LWP with Proxy

LWP Cookbook has the example on how to use proxy for HTTP request.

Here are some more examples:

To connect to an HTTP proxy using LWP, here is what you do:

use LWP;
my $ua = LWP::UserAgent->new;
my $agent = "my-lwp agent";
$ua->agent($agent);
$ua->proxy('http', "http://$proxy_ip:$porxy_port");
# the line below is only needed in some cases that you reuse the same useragent for different proxies.
delete $ENV{HTTPS_PROXY};

If the target website you are visiting via proxy is HTTPS, first make sure you have Crypt::SSLeay installed. Just check it like this:

$ perl -e 'use Crypt::SSLeay'

If it complains, then install Crypt::SSLeay.

After Crypt::SSLeay is installed, the code for setting proxy for https usage is like below:

use LWP;
my $ua = LWP::UserAgent->new;
my $agent = "my-lwp agent";
$ua->agent($agent);
$ENV{HTTPS_PROXY} = "$proxy_ip:$proxy_port";
# the line below is only needed in some cases that you reuse the same useragent for different proxies.
$ua->proxy('http', undef);

Via Crypt::SSLeay, our program will use the "CONNECT" method for https requests. See this document: http://search.cpan.org/~dland/Crypt-SSLeay-0.57/SSLeay.pm#LWP::UserAgent...

LWP::UserAgent has its own methods of proxying which may work for you and is likely to be incompatible with Crypt::SSLeay proxy support. To use LWP::UserAgent proxy support, try something like:
    my $ua = new LWP::UserAgent;
    $ua->proxy([qw( https http )], "$proxy_ip:$proxy_port");
 
At the time of this writing, libwww v5.6 seems to proxy https requests fine with an Apache mod_proxy server. It sends a line like:
    GET https://www.example.com HTTP/1.1
to the proxy server, which is not the CONNECT request that some proxies would expect, so this may not work with other proxy servers than mod_proxy. The CONNECT method is used by Crypt::SSLeay's internal proxy support.
 
Crypt::SSLeay proxy support
For native Crypt::SSLeay proxy support of https requests, you need to set the environment variable HTTPS_PROXY to your proxy server and port, as in:
    # proxy support
    $ENV{HTTPS_PROXY} = 'http://proxy_hostname_or_ip:port';
    $ENV{HTTPS_PROXY} = '127.0.0.1:8080';
 
Use of the HTTPS_PROXY environment variable in this way is similar to LWP::UserAgent-env_proxy()> usage, but calling that method will likely override or break the Crypt::SSLeay support, so do not mix the two.