How to send HTTP request in java?

Question

In Java, How to compose a HTTP request message and send it to a HTTP WebServer?

Solution 1

You can use java.net.HttpUrlConnection.

Example (from here), with improvements. Included in case of link rot:

public static String executePost(String targetURL, String urlParameters) {
  HttpURLConnection connection = null;

  try {
    //Create connection
    URL url = new URL(targetURL);
    connection = (HttpURLConnection) url.openConnection();
    connection.setRequestMethod("POST");
    connection.setRequestProperty("Content-Type", 
        "application/x-www-form-urlencoded");

    connection.setRequestProperty("Content-Length", 
        Integer.toString(urlParameters.getBytes().length));
    connection.setRequestProperty("Content-Language", "en-US");  

    connection.setUseCaches(false);
    connection.setDoOutput(true);

    //Send request
    DataOutputStream wr = new DataOutputStream (
        connection.getOutputStream());
    wr.writeBytes(urlParameters);
    wr.close();

    //Get Response  
    InputStream is = connection.getInputStream();
    BufferedReader rd = new BufferedReader(new InputStreamReader(is));
    StringBuilder response = new StringBuilder(); // or StringBuffer if Java version 5+
    String line;
    while ((line = rd.readLine()) != null) {
      response.append(line);
      response.append('\r');
    }
    rd.close();
    return response.toString();
  } catch (Exception e) {
    e.printStackTrace();
    return null;
  } finally {
    if (connection != null) {
      connection.disconnect();
    }
  }
}

Solution 2

From Oracle's java tutorial

import java.net.*;
import java.io.*;

public class URLConnectionReader {
    public static void main(String[] args) throws Exception {
        URL yahoo = new URL("http://www.yahoo.com/");
        URLConnection yc = yahoo.openConnection();
        BufferedReader in = new BufferedReader(
                                new InputStreamReader(
                                yc.getInputStream()));
        String inputLine;

        while ((inputLine = in.readLine()) != null) 
            System.out.println(inputLine);
        in.close();
    }
}

Solution 3

I know others will recommend Apache's http-client, but it adds complexity (i.e., more things that can go wrong) that is rarely warranted. For a simple task, java.net.URL will do.

URL url = new URL("http://www.y.com/url");
InputStream is = url.openStream();
try {
  /* Now read the retrieved document from the stream. */
  ...
} finally {
  is.close();
}

Solution 4

Apache HttpComponents. The examples for the two modules - HttpCore and HttpClient will get you started right away.

Not that HttpUrlConnection is a bad choice, HttpComponents will abstract a lot of the tedious coding away. I would recommend this, if you really want to support a lot of HTTP servers/clients with minimum code. By the way, HttpCore could be used for applications (clients or servers) with minimum functionality, whereas HttpClient is to be used for clients that require support for multiple authentication schemes, cookie support etc.

Solution 5

Here's a complete Java 7 program:

class GETHTTPResource {
  public static void main(String[] args) throws Exception {
    try (java.util.Scanner s = new java.util.Scanner(new java.net.URL("http://example.com/").openStream())) {
      System.out.println(s.useDelimiter("\\A").next());
    }
  }
}

The new try-with-resources will auto-close the Scanner, which will auto-close the InputStream.

View more solutions

Comments

[Jherico]

That doesn't help if you want to monkey with request headers, something that's particularly useful when dealing with sites that will only respond a certain way to popular browsers.

[brady]

You can monkey with request headers using URLConnection, but the poster doesn't ask for that; judging from the question, a simple answer is important.

[Michael Scheper]

FWIW, our code started with java.net.HttpURLConnection, but when we had to add SSL and work around some of the weird use cases in our screwy internal networks, it became a real headache. Apache HttpComponents saved the day. Our project currently still uses an ugly hybrid, with a few dodgy adapters to convert java.net.URLs to the URIs HttpComponents uses. I refactor those out regularly. The only time HttpComponents code turned out significantly more complicated was for parsing dates from a header. But the solution for that is still simple.

[GreenTurtle]

Here is another nice code snippet in replace for Java Almanac: HttpUrlConnection-Example

[Gorky]

The strange thing is that some servers will reply you back with strange ? characters (which seems like an encoding error related to request headers but not) if you don't open an output stream and flush it first. I have no idea why this happens but will be great if someone can explain why?

[Janus Troelsen]

@Gorky: Make a new question

[Thilo]

What do you mean with 'transport'?

[Tombart]

Sorry, that should have been HTTP_TRANSPORT, I've edited the answer.

[Dan Passaro]

This is way too much line noise to send an HTTP request imo. Contrast to Python's requests library: response = requests.get('http://www.yahoo.com/'); something of similar brevity should be possible in Java.

[Joffrey]

Seriously, I really like Java, but what's the matter with that stupid NameValuePair list or array. Why not a simple Map<String, String>? So much boilerplate code for such simple use cases...

[Cypher]

Putting some actual code into this answer will help avoid link rot...

[fortran]

@leo-the-manic that's because Java is supposed to be a lower level language (than python) and allows (forces) the programmer to handle the details underneath rather than assuming "sane" defaults (i.e. buffering, character encoding, etc.). It is possible to get something as succinct, but then you lose lots of the flexibility of the more barebones approach.

[Felype]

@leo-the-manic If there is no such thing as you look for, do go and write by yourself. I do that all the time and I do have quick get/post static methods on both java and C#

[Janus Troelsen]

why is HttpResponse not AutoClosable? What is the difference from this and to working with Apache's CloseableHttpClient?

[Jeff Fairley]

The benefit is the API, which makes it personal preference really. Google's library uses Apache's library internally. That said, I like Google's lib.

[CuriousGuy]

@laksys why it should be \r\n instead of \n?

[laksys]

@CuriousGuy look at this link programmers.stackexchange.com/questions/29075/…

[Ben]

@Joffrey Maps by definition have 1 key per value, means: A map cannot contain duplicate keys ! But HTTP Parameters can have duplicate keys.

[User]

@fortran Python has equally low-level options to accomplish the same thing as above.

[hoodaticus]

"that's because Java is supposed to be a lower level language" X'D

[Ska]

That's exactly why Java is stopped being thought in schools. Relic of the history.

[DanGordon]

@fortran Java is not 'lower level' than Python... how does that make any sense?

[jerzy]

@Ska There is no unhandled exception. main() throws Exception, which encompasses the MalformedURLException and the IOException.

[SwiftMango]

It seems to be even easier and more straight forward than the other solutions. Java makes things more complicated than it should be.

[Baruch Atta]

What Dan Passaro is saying is there should be a wrapper for all that code. And the wrapper should be part of the Plain Old Java Objects. So he (and me) don't need to worry about all of the under-the-hood mechanics. That would be a class that has, as input, the URL, and as output, the String [] array. or just one long string.

[Baruch Atta]

my wrapper: // import java.net.*; // import java.io.*; public static String getWebPage(String targetURL) { String s = "", t = ""; URL url = null; URLConnection connection = null; try { url = new URL(targetURL); connection = url.openConnection(); connection.connect(); BufferedReader in = new BufferedReader(new InputStreamReader( connection.getInputStream())); while ((t = in.readLine()) != null) { s = s + "\n" + t; } in.close(); } catch (IOException e) { e.printStackTrace(); } return s; }

[WesternGun]

Scanner actually is not very optimized when it comes to performance.

[Vic Seedoubleyew]

It would be helpful to add a code snippet here

[Anton Sorokin]

Since Java 9, creating HTTP request has become much easier.

[duffymo]

Yes, a lot has changed in the ten years since this answer was given. Not everyone has moved on from JDK8 to 9 and beyond.