Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

http_proxy and db cache issue #108

Open
ykelvis opened this issue May 14, 2024 · 0 comments
Open

http_proxy and db cache issue #108

ykelvis opened this issue May 14, 2024 · 0 comments

Comments

@ykelvis
Copy link
Contributor

ykelvis commented May 14, 2024

It seems if the proxy server is unavailable (either offline completely or due to network issue) when pacoloco is actively using it, then pacoloco will become completely unusable. Only restart can fix.
And also seems pacoloco don't send locally available cache (for db files) when network is down.

Basic info:

Platform: docker
Image: ghcr.io/anatol/pacoloco:latest

Steps to reproduce:

  1. Configure pacoloco to use http_proxy, http_proxy: http://192.168.0.11:8080
  2. Start a proxy server at port 8080, and start pacoloco
  3. Run pacman -Syu. Everything works fine.
  4. Kill the proxy server.
  5. Run pacman -Syu, pacman returns "failed to download".
  6. Start the proxy server again. Expect everything goes back to normal.
  7. Run pacman -Syu, but pacman still returns same network error. I also noticed that there is no requests made to the proxy server.
  8. Restart pacoloco container, everything works again.

The errors in step 7:

On pacman:

:: Synchronizing package databases...
 core.db failed to download
 extra.db failed to download
 multilib.db failed to download
 archlinuxcn.db failed to download
error: failed retrieving file 'core.db' from mirrors.local : The requested URL returned error: 404
error: failed retrieving file 'extra.db' from mirrors.local : The requested URL returned error: 404
error: failed retrieving file 'multilib.db' from mirrors.local : The requested URL returned error: 404
warning: too many errors from mirrors.local, skipping for the remainder of this transaction
error: failed retrieving file 'archlinuxcn.db' from mirrors.local : The requested URL returned error: 404
error: failed to synchronize all databases (failed to retrieve some files)

On pacoloco:

pacoloco-1  | 2024/05/14 21:27:26 unable to download file archlinux/extra/os/x86_64/extra.db
pacoloco-1  | 2024/05/14 21:27:26 unable to download file archlinux/multilib/os/x86_64/multilib.db
pacoloco-1  | 2024/05/14 21:27:26 unable to download file archlinuxcn/x86_64/archlinuxcn.db
pacoloco-1  | 2024/05/14 21:27:26 unable to download file archlinux/core/os/x86_64/core.db

Prefetch is also broken:

pacoloco-1  | 2024/05/14 21:36:00 Starting prefetching packages...
pacoloco-1  | 2024/05/14 21:36:00 An error occurred for mirror {/repo/archlinux/core/os/x86_64/core.db archlinux 2024-05-14 21:32:21.846675592 +0800 +0800} :unable to download file archlinux/core/os/x86_64/core.db
pacoloco-1  | 2024/05/14 21:36:00 An error occurred for mirror {/repo/archlinux/extra/os/x86_64/extra.db archlinux 2024-05-14 21:32:23.907249368 +0800 +0800} :unable to download file archlinux/extra/os/x86_64/extra.db
pacoloco-1  | 2024/05/14 21:36:00 An error occurred for mirror {/repo/archlinux/multilib/os/x86_64/multilib.db archlinux 2024-05-14 21:32:20.908618101 +0800 +0800} :unable to download file archlinux/multilib/os/x86_64/multilib.db
pacoloco-1  | 2024/05/14 21:36:00 An error occurred for mirror {/repo/archlinuxcn/x86_64/archlinuxcn.db archlinuxcn 2024-05-14 21:32:22.503971676 +0800 +0800} :unable to download file archlinuxcn/x86_64/archlinuxcn.db
pacoloco-1  | 2024/05/14 21:36:00 Finished prefetching packages!
pacoloco-1  | 2024/05/14 21:36:00 Finished prefetching routine!
pacoloco-1  | 2024/05/14 21:36:00 On 2024-05-14 21:38:59.999999953 +0800 CST m=+404.298213542 the prefetching routine will be run again
@ykelvis ykelvis mentioned this issue May 16, 2024
Merged
anatol pushed a commit that referenced this issue May 29, 2024
@ykelvis
Fix network issue and use cache when network is down (#109)
bb774b2 
Relate to #108

I think it's not only an issue with http_proxy. I can reproduce this error without http_proxy settings by plug/unplug my network cable (forcing a connection issue during pacoloco's network activity), same problem will show up.

After a little tinkering I think the problem is d is not ended by default. And I added f.cachedFileExists so that cache can be served. (Otherwise pacoloco would just return err without sending anything. https://github.com/anatol/pacoloco/blob/master/pacoloco.go#L270)

This has a little bit downside, it would not register a cacheServingFailedCounter if cache is sent. But I guess that would not be a fail after all?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ykelvis