Login to a website using HTTP POST in Java

Introduction

Did you know that you can browse through a website using a Java program that you write? The Java SDK provides rich functional classes that will allow you to write a fully functional web browser. While writing a web browser is an elaborate task, this article attempts to get you started with the task. This article provides the basic concepts needed to connect to a remote URL, and POST some data to it.

Before we proceed to the concepts and code, let me first define a scenario that will help us understand the concepts clearly. Lets say, we want to write a Java program that can login to a remote web site (a web site that is out there in the deep and dark Internet!). For this, we need the following inputs

• The URL of the login page of the web site. (String TARGET_URL)
• The username for login (String LOGIN_USER_NAME)
• the password for login (String LOGIN_PASSWORD)

Note that the content within brackets can be used as a cross reference to the actual sample code that can be downloaded at the end of this article.

Preparing the POST message:

The first thing we need to do is to prepare the login request message. Note that this message is specific to the login method and script of the target web site. For the sample login script provided at the end of this article, the login request message can be constructed as follows

String encodedLoginUserName = URLEncoder.encode(loginUserName, "UTF-8");
String encodedLoginPassword = URLEncoder.encode(loginPassword, "UTF-8");
String content = "login=" + LOGIN_ACTION_NAME +" &" + 
                 LOGIN_USER_NAME_PARAMETER_NAME +"=" + encodedLoginUserName + 
                 "&" + LOGIN_PASSWORD_PARAMETER_NAME + "=" + encodedLoginPassword;

Note that the username and password are encoded to the UTF-8 format. For more information on this please see this link.

Creating a URL object:

The URL class defined in the Java 2 SDK is used to represent a URL (Uniform Resource Locator) which is a pointer to a resource in the internet. In our case, this resource is the login page of the website we want to login to.

So the first step in our program is to create a URL object for the target URL which can simply be done by passing the target URL as a parameter to the URL object's constructor while instantiating it.

URL urlObject = new URL(targetUrl);

Once this URL object is created, a remote connection can be created by simply calling its openConnection() method. This method openConnection() is intelligent enough to automatically determine the protocol used i.e. whether it is HTTP, FTP etc. using the specified targetURL and return an appropriate specialised URLConnection instance, based on the protocol. In our case, the target URL will start with "http://www." since it targets a webpage and so the specialised URLConnection instance will be of type HttpURLConnection.

Configuring the HttpURLConnection class:

This specialised HttpURLConnection class provides certain methods specific to the HTTP protocol. With the instance of this class acquired, we are now almost ready to do the HTTP POST with only some configuration to be done. The following are the configurations to the HttpURLConnection to be done...

//Specifying that we intend to use this connection for input
    urlConnection.setDoInput(true);
 
// Specifying that we intend to use this connection for output
    urlConnection.setDoOutput(true);
 
// Specifying the content type of our post
    urlConnection.setRequestProperty("Content-Type", POST_CONTENT_TYPE);
 
// Specifying the method of HTTP request which is POST
    // throws ProtocolException
    urlConnection.setRequestMethod("POST");

Writing the POST message using DataOutputStream class:

For us to write the POST message, we must get the output stream of the HttpURLConnection which can then be wrapped with by a DataOutputStream object. The DataOutputStream class is used to write primitive Java type to an output stream which in our case is going to be a String.

dataOutputStream = new DataOutputStream(urlConnection.getOutputStream());
 
// With the DataOutputStream object, the POST contents can simply be written using the folowing code.
 
dataOutputStream.writeBytes(content);
dataOutputStream.flush();
 
// And finally don't forget to close the output stream to free up any resources used.
 
dataOutputStream.close();

Reading the response using the BufferedReader class:

Finally we may want to read back the results of our login request. This can be done by getting the input stream of the HttpURLConnection object wrapping it in a BufferedReader object. The BufferedReader class is very efficient for reading characters which is what we are expecting.

// Prepare a reader to read the response from the URLConnection
    // throws IOException
 
bufferedReader = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
String responeLine;
 
// Good Practice: Use StringBuilder in this case
StringBuilder response = new StringBuilder();
 
// Read untill there is nothing left in the stream
// throws IOException
while ((responeLine = bufferedReader.readLine()) != null)
{
   response.append(responeLine);
}
return response.toString();

A complete listing of the class that has all of the above discussed functionality and a download link are available in the next page of this article. Furthermore you can use this code to run against a sample login PHP script available at this link. This PHP script is also available for download if you want to run it locally. Again check out the next page of this article to download.