Detect mobile devices from user agent string

Question

I am looking for a way to analyze user agent strings to determine whether they were generated by mobile devices. This needs to be java-based and usable in large batch log file analysis on hadoop for generating statistics (i.e., a web service wouldn't be appropriate).

I have seen WURFL, but given that I just need a binary mobile/not mobile response, the license fee seems prohibitive.

So far I have been using UADetector, which is almost exactly what I need. However, I have encountered some limitations with it. In my testing, I have found many user agent strings that provide enough information to determine that the user agent is from a mobile device, but are reported as UNKNOWN by UADetector.

For example, poorly-standardized Android apps can send the UA string "Android". This is enough to know that it came from a mobile device, but UADetector reports this UserAgentType as UNKNOWN rather than MOBILE_BROWSER.

Apache Mobile Filter's Lite Device Detection does the right thing, but I need something I can use from Java.

Can anyone recommend a better solution?

Solution 1

I'm the founder and maintainer of the MobileESP project, a free open source cross-platform library for detecting mobile devices. It's still very much alive! :-)

www.mobileesp.org

MobileESP only gives binary "is mobile" responses. You can detect by platform like iOS, Android or Windows Phone, or by device category, like "iPhone Tier" smartphones vs. tablet. Be sure to take a quick review of the API page.

As you may know, useragent strings vary widely. If the browser shipped on the device, the manufacturer may customize it. For example, HTC often customizes the native Android browser's useragent string.

Google provides recommendations on how the OEM should customize the useragent. If the device should be considered a phone, then Google recommends including the word "mobile" element in the string. But if the device should be considered a tablet, then the string should not contain "mobile." Adherence to this recommendation varies widely, of course.

Third party browsers like Opera or Maxthon can put whatever they want to in the useragent string -- and do! Certain "new" browsers which shall remain nameless have been doing very poor jobs of putting the correct information in their useragent strings for each platform (e.g., Android vs. iOS versions). There's not much you can do unless you get a lot of traffic from these browsers and wish to invest in tracking their exact useragent values per platform and software rev.

Anyway, MobileESP was created with the vision of doing the detection on a page-by-page basis when the page is served. I purposefully wrote the code to be very easy to read and customize, too.

To do the batch processing, you might do something like this:

1.) In the constructor, comment out the initDeviceScan() method. You won't need this for bulk processing.

2.) Pass the UserAgent and an empty string in to the constructor (UAgentInfo()).

3.) Then run whatever detect methods you're interested in. Be thoughtful about the order in which you do them to save time, based on a scan of your users.

For example, if most of your users are on iPhone and that's one of the detection criteria you're interested in, then run that check first. If this example, you certainly wouldn't run the BlackBerry method first!

My contact info is in the source code and on the web site. Send me a note if you have any questions or run into any bugs. Definitely look around the MobileESP.org web site for some tips.

Best wishes on your project, Aniket!

• Anthony

Solution 2

Another thread suggests using the following library:

https://github.com/ahand/mobileesp/blob/master/Java/UAgentInfo.java

which seems OK.

Solution 3

51Degrees has a free open source Java API that allows you to run offline processing. You can access it from the GitHub Repository here. https://github.com/51Degrees/Java-Device-Detection.

As part of the API there is an offline processing example (code also shown below) this takes a CSV file of User-Agents and returns the required properties into an Output file. The following example just uses 3 of the properties within the data set, for a full list you can look at the dictionary here https://51degrees.com/resources/property-dictionary

// output file in current working directory
public String outputFilePath = "batch-processing-example-results.csv";
// pattern detection matching provider
private final Provider provider;

/**
 * Initialises the device detection Provider with the included Lite data
 * file. For more data see: 
 * <a href="https://51degrees.com/compare-data-options">compare data options
 * </a>
 * 
 * @throws IOException if there was a problem reading from the data file.
 */
public OfflineProcessingExample() throws IOException {
    provider = new Provider(StreamFactory.create(
            Shared.getLitePatternV32(), false));
 }

/**
 * Reads a CSV file containing User-Agents and adds the IsMobile, 
 * PlatformName and PlatformVersion information for the first 20 lines.
 * For a full list of properties and the files they are available in please 
 * see: <a href="https://51degrees.com/resources/property-dictionary">
 * Property Dictionary</a>
 * 
 * @param inputFileName the CSV file to read from.
 * @param outputFilename where to save the file with extra entries.
 * @throws IOException if there was a problem reading from the data file.
 */
public void processCsv(String inputFileName, String outputFilename) 
        throws IOException {
    BufferedReader bufferedReader = 
            new BufferedReader(new FileReader(inputFileName));
    try {
        FileWriter fileWriter = new FileWriter(outputFilename);
        try {
            // it's more efficient over the long haul to create a match 
            // once and reuse it in multiple matches
            Match match = provider.createMatch();
            // there are 20k lines in supplied file, we'll just do a couple 
            // of them!
            for (int i = 0; i < 20; i++) {

                // read next line
                String userAgentString = bufferedReader.readLine();

                // ask the provider to match the UA using match we created
                provider.match(userAgentString, match);

                // get some property values from the match
                Values isMobile = match.getValues("IsMobile");
                Values platformName = match.getValues("PlatformName");
                Values platformVersion = match.getValues("PlatformVersion");

                // write result to file
                fileWriter.append("\"")
                        .append(userAgentString)
                        .append("\", ")
                        .append(getValueForDisplay(isMobile))
                        .append(", ")
                        .append(getValueForDisplay(platformName))
                        .append(", ")
                        .append(getValueForDisplay(platformVersion))
                        .append('\n')
                        .flush();
            }
        } finally {
            fileWriter.close();
        }
    } finally {
        bufferedReader.close();
    }
}

/**
 * Match values may be null. A helper method to get something displayable
 * @param values a Values to render
 * @return a non-null String
 */
protected String getValueForDisplay(Values values) {
    return values == null ? "N/A": values.toString();
} 

/**
 * Closes the {@link fiftyone.mobile.detection.Dataset} by releasing data 
 * file readers and freeing the data file from locks. This method should 
 * only be used when the {@code Dataset} is no longer required, i.e. when 
 * device detection functionality is no longer required, or the data file 
 * needs to be freed.
 * 
 * @throws IOException if there was a problem accessing the data file.
 */
@Override
public void close() throws IOException {
    provider.dataSet.close();
}

/**
 * Instantiates this class and starts 
 * {@link #processCsv(java.lang.String, java.lang.String)} with default 
 * parameters.
 * 
 * @param args command line arguments.
 * @throws IOException if there was a problem accessing the data file.
 */
public static void main(String[] args) throws IOException {
    System.out.println("Starting Offline Processing Example");
    OfflineProcessingExample offlineProcessingExample = 
            new OfflineProcessingExample();
    try {
        offlineProcessingExample.processCsv(Shared.getGoodUserAgentsFile(), 
                offlineProcessingExample.outputFilePath);
        System.out.println("Output written to " + 
                offlineProcessingExample.outputFilePath);
    } finally {
        offlineProcessingExample.close();
    }
}

Hope this helps.

Disclosure: I work at 51Degrees.

Solution 4

How to read the Apache Mobile Filter value in JSP (for Tomcat)?

Before in the httpd.conf file where you have to configure mod_jk you muse add this:

JkEnvVar AMF_IS_MOBILE undefined

The Java code is:

request.getAttribute("AMF_IS_MOBILE")

from: http://wiki.apachemobilefilter.org

View more solutions

Comments

[Eatsam ul haq]

Hi @Anthony I am using mobileesp.org in my Java Spring Boot project. I wanted to know, will that also work if I change my browser screen width (from inspect element)? And secondly, I am always getting false on all kinds of mobile and table detection methods.