Chrome extension unable to get data from a server after switch to https due to Origin Control

Question

I've been battling with Chromium, jQuery and AJAX. My extension takes some resources from some sites using ajax, then parse the results and store it somewhere. One of those sites decided to do the jump to https. Neat. I just have to change the http for https, and fix any problem I hit down the road. No joy.

I use the following ajax method to get the stuff (is the smallest unit I could reproduce):

$.ajax({
        url: "https://reader.japanzai.com/search/",
        type: 'POST',
        data: {
            'search': "onna"
        },
        beforeSend: function (xhr) {
            xhr.setRequestHeader("Access-Control-Allow-Origin", "*");
        },
        headers: {
            'Access-Control-Allow-Origin': '*'
        }})

Lets ignore for a while that I've set the headers twice, since just using one don't work either and throw the result:

OPTIONS https://reader.japanzai.com/search/ No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'chrome-extension://nhjloagockgobfpopemejpgjjechcpfd' is therefore not allowed access. jquery.js:5
x.support.cors.e.crossDomain.send jquery.js:5
x.extend.ajax jquery.js:5
(anonymous function) VM4000:2
InjectedScript._evaluateOn VM3770:581
InjectedScript._evaluateAndWrap VM3770:540
InjectedScript.evaluate VM3770:459
XMLHttpRequest cannot load https://reader.japanzai.com/search/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'chrome-extension://nhjloagockgobfpopemejpgjjechcpfd' is therefore not allowed access. lab.html:1

As you can see, it says that the request header is not present, which is true following the http request:

Request URL:https://reader.japanzai.com/search/
Request Method:OPTIONS
Status Code:200 OK
Request Headersview parsed
OPTIONS https://reader.japanzai.com/search/ HTTP/1.1
origin: chrome-extension://nhjloagockgobfpopemejpgjjechcpfd
access-control-request-method: POST
dnt: 1
accept-encoding: gzip,deflate,sdch
host: reader.japanzai.com
accept-language: es,en-US;q=0.8,en;q=0.6
method: OPTIONS
user-agent: Mozilla/5.0 (X11; Linux i686 (x86_64)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36
accept: */*
url: /search/
scheme: https
version: HTTP/1.1
access-control-request-headers: access-control-allow-origin, accept, content-type

Response:

HTTP/1.1 200 OK
cache-control: post-check=0, pre-check=0, max-age=0
content-encoding: gzip
content-type: text/html
date: Thu, 13 Feb 2014 22:58:03 GMT
expires: Sat, 01 Jan 2000 00:00:01 GMT
last-modified: Thu, 13 Feb 2014 22:58:03 GMT
pragma: no-cache
server: nginx/1.5.8
set-cookie: ci_session=U3Q14aq8Q7W4KVifDyLi7%2B3lppr%2FS4BNmW1kD9t60H7dz73M5KMs1jgBo8ZrilKoswn63PvCl1sOldPs1RCl6NdiP1VZeekoMTISyQenQZU0F8yUC0odw6WuMUE5I%2FY%2Bfvddv2YH06b2pxyyNBDZnNn%2BLnLzPrRYBXHuAjFbTyucX%2FMLUUM2cwKLzaK3ILKPhxy8FXW%2FI%2F9EPPbwo%2B8nmbPwOeqDfpDdu61F5yzUU8KjfUo7MwwFIXyGWtqbjbF3PCKNZrY%2F3Cj77DgCpcCbTTKZ%2BVzrdw16oGVDg1dP8lQgSof89rLNqUlQSj60tCVzZ27oPNh9OvvTNJ92tYkTHDukG4dyv21yM4M3PACZ%2FKVNP0i2UWHbBujADPSsrGJhJxPzBsuRDLcPtDcBtuaXA4LLMoGoYW6SxYk%2BseltMvk%3D; expires=Fri, 14-Feb-2014 00:58:03 GMT; path=/
set-cookie: ci_session=tQT8qmNRnMRN2Oj3moCdZg9VNEEsPxi3t88g2SpYQxahFr%2FpiEpQFzsO2mLTp1bPlsGLmqQGnMUiuwFpLYNIneNHtU%2BoKkVOcnR8ZKxPd0FDrkW%2BqT0N2IIsV%2BC%2FXQX%2BZUkLg1E4iP6u%2F0%2Fjk1t%2BAwcwhoC0M3zODuEKv1l9JMFo%2B1g4%2BhIOp%2FHTzBnlMvE2KjanXJR55F3DOHdyi4MvQb1vzgWEZTTAfhZ3bkQPkKe41ZCJYQTw%2FrDfry8n2h43UKPc1IF4tWp%2BKh0yhux%2FsBn84meT3xR%2Bpba9ffeZObrQyVomKlmJg9oRkKvlhR4MlNsiIeIZEvtP52ns0X1uF%2B7Pg6RpcMihe1u2S0%2Fbz5wm75vQ6tyykmFp5qfnoDgXB6J7RmbBQy4GTOFEA2zqN3V6QXT71cSn%2B1ARd9GtNMA%3D; expires=Fri, 14-Feb-2014 00:58:03 GMT; path=/
status: 200 OK
strict-transport-security: max-age=31536000
vary: Accept-Encoding
version: HTTP/1.1
x-powered-by: PHP/5.4.4-14+deb7u7

So, I'm missing something obvious here or there's just no way to do this?

Solution

I followed abraham advice and added the site to the permissions field in the manifest. It worked, the only bad thing is that if another site decides to move to https I need to release a new version, so this is what I ended doing:

  "permissions" : [
    "tabs",
    "*://*/*",
    "https://ssl10.ovh.net/*",
    "unlimited_storage",
    "clipboardWrite",

The important part here is "*://*/*" which includes anything from http and https.

Comments

[Rob W]

Don't ask for permissions if you do not need it. If the problem is merely a switcb from http to https, replace http with a wildcard, i.e. use "*://ssl10.ovh.net/*". This wilcard matches http and https only, and using the wildcard instead of http does not lead to additional permission warnings. If you add <all_urls>, on the other hand, then the extension will be disabled until the user approves the new permissions. Even if you really want to access all sites, use "*://*/*" instead of "<all_urls>".

[Rob W]

(continued) because "<all_urls>" usually matches more than you need. Besides http(s), it also includes ftp:// and file://. If the command flag permits, it will also match chrome://. In the future, it will also match about://. So, "<all_urls>" should not be used unless you expect to need these permissions. *://*/* is often more than enough. (Both permissions trigger the same permission warning, so if you ever need to access ftp for some reason, then you can easily switch to <all_urls>.)

[Braiam]

@RobW thanks, it seems to work fine the "*://*/*".

[Xan]

@Braiam While it works fine, your users will be presented with a scary permissions warning on install and will rightfully shy away from it.

[Braiam]

@Xan don't worry, my original had http://*/*, and if anything I will add a release note.