How can I retrieve the HTML for a URL and write it to a file?

Solution 1: LWP::Simple

The LWP::Simple module provides a simple interface to the LWP (lib-www-perl) modules. The example below will retrieve a web page and write the HTML to the local file "some_local_file":

    #!/usr/bin/perl
    use strict;
    use warnings;
    use LWP::Simple;

    getstore('http://www.perlmeme.org', 'some_local_file');
    exit 0;

Solution 2: LWP

The LWP object oriented style allows for more flexible behaviour:

    #!/usr/bin/perl
    use strict;
    use warnings;
    use LWP::UserAgent;

    my $ua = LWP::UserAgent->new;
    $ua->agent("Mozilla/8.0");  # Pretend to be Mozilla

    my $req = HTTP::Request->new(GET => 'http://www.perlmeme.org');

    # The second parameter to request can be a filename
    my $res = $ua->request($req, 'pmstore');

    if ($res->is_success) {
        print "Success!\n";
    } else {
        print "Error: " . $res->status_list . "\n";
    }

    exit 0;

Solution 3: Command Line

Once you have installed the LWP module, you can get pages on the command line as follows:

    GET http://www.perlmeme.org > pmstore

See Also

    perldoc LWP::Simple
    perldoc lwpcook
    perldoc lwptut
    perldoc LWP::UserAgent