Installing gems from behind a corporate firewall

Question

I suspect that the corporate firewall is preventing gems from getting installed. I have HTTP_PROXY defined and I'm able to view remote gems via the following command:

jruby -S gem list -r

But when I go to install a gem, I get a 404:

jruby -S gem install rails

Is there a good workaround for resolving this issue other than maintaining an internal gem repository?

Solution 1

I've retested this since the gemcutter.org folks made the mode to Amazon's Cloudfront and am able to download gems again through my employer's firewall.

Solution 2

for ruby gems, placing this in my gem.bat this works for me

@"%~dp0ruby.exe" "%~dpn0" %* --http-proxy http://domainname.ccc.com:8080

for jruby gems this works

@"%~dp0jruby.exe" "%~dpn0" %* -p http://domainname.ccc.com:8080

Also setting the environment variable works like

set http-proxy=http://domainname.ccc.com:8080

or if your proxyserver needs authentication

set http-proxy=http://user:password@host:port)

EDIT: for folks who have a very restrictive firewall or no internet accesss (eg on a server) you can do the following: install the gem(s) on a pc who has free access to internet, afterward you check the folder C:\Ruby193\lib\ruby\gems\1.9.1\cache and copy all the gems with a date last modified after the moment you did your install. On the target pc you copy them in the same folder or if you clean up afterward in your bin folder and start the install there with

c:\ruby193\bin\gem.bat install --local gemname-x.x.x.gem

gemname-x.x.x.gem being the gem filename of the base gem you want to install. The gem with all its dependencies should install. This is for windows, but other OS's can use the same technique, just adapt a few things. Success

Solution 3

1. Look in Internet Explorer proxy settings to find the name of your proxy server.

2. Use the name of your proxy server in the -p option to the gem command.

   gem update rails -p http://mylocalproxy.mycompany.com

If this doesn't work, you can always set your own proxy server up on the internet somewhere.

Solution 4

I'm not an expert with Ruby. None the less I might give the following "generic" advices (useful for lots of situations with installers) :

1. Check which protocol is the installation running over. Some installer download package from mirror instead of main site, and some mirrors might be using FTP or something else instead of HTTP. Thus you might need to update your settings accordingly, and check with your admin that the corporate proxy is able to retrieve data from FTP sites.

2. Check which user the installer is running as (some of the installer have to escalate privileges to "root". And therefore inherit a different environment with another HTTP_PROXY defined).

3. Check what the installer program is using to download the packages (lots of distribution package managers rely on curl/libcurl which stores its proxy parameters in a rc file). Same warning as 2 regarding the users whose homedir should contain the rc file.

4. Transparent proxying : it is possible to transfer on-the-fly HTTP/FTP requests initially targeting the web to a proxy. Either the corporate proxy (if it can work with such an installation) or some mini local proxy which will then cascade to the corporate one.

5. Setup a quick VPN (either Tunnel other PPP) over SSH (see SSH's man page) between your machine and a special machine within the corporation which is allowed to see the web, then update your routing tables to route traffic through the VPN.

6. You can also use corkscrew to connect through SSH (and thus be able to install a VPN over SSH like 5) with a machine outside, on the other side of the corporate firewall, like your own at home. (this uses the HTTPS "CONNECT" mode of the proxy to access a SSH outside). Just make sure that the admins are- or at least one person in charge is- kept in loop and don't panic.

7. Desperate measure : create a tunnel between two machines running each http tunnel, over a proxy which only allows HTTP requests. Then use the tunnel to establish SSH VPN.

These solutions aren't specific to Ruby but can help you through any desperate solution behind an over-restrictive firewall.

1 to 4 should work in most situation. You'll have to resort to 5 to 7 if confronted with some bizarre and/or paranoid firewalls. 7 is rather extreme but always works, although sometimes with catastrophic performance.

Solution 5

Simple thing first: are you in the same shell session when you tried both the list and the install? Maybe you forget to set the env second time around?

Try setting http_proxy instead of HTTP_PROXY.

Make sure the proxy setting is a valid URL (see below)

Do you have multiple proxy options? Open IE and type wpad in the address bar. You might see some other possibilities.

Is your proxy authenticated? If so, do you have your credentials in the URL. Mine is like this (on windows):

set http_proxy=http://myuserid:mypassword@internetproxy:3128

Does your your password have any punctuation characters? Try it with just numbers and letters, as : and @ (and possibly others) are significant in the URL string itself.

View more solutions

Comments

[digitalsanctum]

I've tried all of these suggestions already and the proxy is not authenticated.

[digitalsanctum]

Proxying and/or a tunnel really isn't an option here. The corporate firewall is there for a reason and I don't want to risk my job by circumventing it.

[DrYak]

You can still proxy TO the corporate proxy in a 2-stage implementation, in case the corporate proxy happens to be incompatible with the software. (The intermediate proxy is used to relay requestion between the two incompatible pieces of software). Also, a SSH-based tunnel (either SSH-VPN or SSH-PPP) can be made really secure by using a correct authentication scheme and remote machine. Also the tunnel is alive only while SSH is running. Thus could be turned on and off as part of an update script. "risking your job" : didn't I mention keeping the admins or the person in chage informed ?

[Ian Vaughan]

Did you mean to use "%3A" in "myuserid%3Amypassword", or should it be a colon ":"?

[Robert Brown]

Yes its a colon, Stack Overflow is encoding it for some reason.

[Robert Brown]

@IanVaughan - thanks - change backticks to 4-space-indent fixed it

[Cocoa Dev]

where do you type that or add it?

[Cocoa Dev]

Whats the location of gem.bat

[Cocoa Dev]

Whats the location of gem.bat? I have two copies. One is in C:\Ruby193\bin and the other is in C:\RailsInstaller\Ruby1.9.3\bin