How can I pass variable into an evaluate function?

Question

I'm trying to pass a variable into a page.evaluate() function in Puppeteer, but when I use the following very simplified example, the variable evalVar is undefined.

I'm new to Puppeteer and can't find any examples to build on, so I need help passing that variable into the page.evaluate() function so I can use it inside.

const puppeteer = require('puppeteer');

(async() => {

  const browser = await puppeteer.launch({headless: false});
  const page = await browser.newPage();

  const evalVar = 'WHUT??';

  try {

    await page.goto('https://www.google.com.au');
    await page.waitForSelector('#fbar');
    const links = await page.evaluate((evalVar) => {

      console.log('evalVar:', evalVar); // appears undefined

      const urls = [];
      hrefs = document.querySelectorAll('#fbar #fsl a');
      hrefs.forEach(function(el) {
        urls.push(el.href);
      });
      return urls;
    })
    console.log('links:', links);

  } catch (err) {

    console.log('ERR:', err.message);

  } finally {

    // browser.close();

  }

})();

Solution 1

You have to pass the variable as an argument to the pageFunction like this:

const links = await page.evaluate((evalVar) => {

  console.log(evalVar); // 2. should be defined now
  …

}, evalVar); // 1. pass variable as an argument

You can pass in multiple variables by passing more arguments to page.evaluate():

await page.evaluate((a, b c) => { console.log(a, b, c) }, a, b, c)

The arguments must either be serializable as JSON or JSHandles of in-browser objects: https://pptr.dev/#?show=api-pageevaluatepagefunction-args

Solution 2

I encourage you to stick on this style, because it's more convenient and readable.

let name = 'jack';
let age  = 33;
let location = 'Berlin/Germany';

await page.evaluate(({name, age, location}) => {

    console.log(name);
    console.log(age);
    console.log(location);

},{name, age, location});

Solution 3

Single Variable:

You can pass one variable to page.evaluate() using the following syntax:

await page.evaluate(example => { /* ... */ }, example);

Note: You do not need to enclose the variable in (), unless you are going to be passing multiple variables.

Multiple Variables:

You can pass multiple variables to page.evaluate() using the following syntax:

await page.evaluate((example_1, example_2) => { /* ... */ }, example_1, example_2);

Note: Enclosing your variables within {} is not necessary.

Solution 4

It took me quite a while to figure out that console.log() in evaluate() can't show in node console.

Ref: https://github.com/GoogleChrome/puppeteer/issues/1944

everything that is run inside the page.evaluate function is done in the context of the browser page. The script is running in the browser not in node.js so if you log it will show in the browsers console which if you are running headless you will not see. You also can't set a node breakpoint inside the function.

Hope this can help.

Solution 5

For pass a function, there are two ways you can do it.

// 1. Defined in evaluationContext
await page.evaluate(() => {
  window.yourFunc = function() {...};
});
const links = await page.evaluate(() => {
  const func = window.yourFunc;
  func();
});


// 2. Transform function to serializable(string). (Function can not be serialized)
const yourFunc = function() {...};
const obj = {
  func: yourFunc.toString()
};
const otherObj = {
  foo: 'bar'
};
const links = await page.evaluate((obj, aObj) => {
   const funStr = obj.func;
   const func = new Function(`return ${funStr}.apply(null, arguments)`)
   func();

   const foo = aObj.foo; // bar, for object
   window.foo = foo;
   debugger;
}, obj, otherObj);

You can add devtools: true to the launch options for test

View more solutions

Comments

[chitzui]

Hello, how would you pass multiple variables?

[chitzui]

Also, I’m not really able to pass a function: var myFunction = function() { console.log("hello") }; await page.evaluate(func => func(), myFunction); gives me: Evaluation failed: TypeError: func is not a function.. Why?

[nanquim]

And I wanted to pass an object?

[user3568719]

How would you add an argument in the 2nd case? eg I want to add pass a string to yourFunc

[wolf]

You can replace yourFunc with object If your property is not a function. @tramada

[wolf]

func is similar to youFunc，so you can call func(stringArg) just like exec yourFunc @user3568719

[wuno]

Would you mind showing how you would pass an object to window and then access it?

[wolf]

@wuno Just like the second example.

[Flimm]

Don't forget to type evalVar both in the function argument signature, and as a passed argument to evaluate (at the end of the code example).

[knod]

@chitzui: You can't pass a function to pate.evaluate(). You can supposedly 'expose' it with page.exposeFunction. For more, see stackoverflow.com/a/58040978.

[avalanche1]

How do you run puppeteer in typescript? Do you transpile to js each time you modify your code?

[Srinivas Reddy Thatiparthy]

Yes. You can take a look at this project here - github.com/srinivasreddy/companies-list

[Mix Master Mike]

since this has the highest upvotes, has anyone experienced linting errors with this? specifically with the parameter already declared in the upper scope. aside from this error, this does work.

[Erwol]

Your answer helped me to figure out how to pass parameters to $eval: page.$eval(selector, (element, attribute) => element.getAttribute(attribute), attribute);.

[Srinivas Reddy Thatiparthy]

@Erwol glad to be of help.

[ggorlen]

It's not clear to me that this is any more convenient or readable than passing the vars directly as shown here. This just allocates and destructures an object, increasing garbage collection activity and adding more braces. It's not a big deal but not much of an improvement, either, when the primary issue is that OP isn't passing parameters at all.

[Mehdi Raash]

Disagree, allocating and destruction of array is a big deal in this case, however VM optimizes it. The plus point of this style is that you don't care about the sequence of the variable insertion. so you can retrieve the variables(properties) as you wish.

[Mehdi Raash]

* of objects is not a big deal

[ggorlen]

I hadn't thought of the reordering, but that seems like a rather pointless characteristic--it strikes me as less intuitive to call a function like fn(({baz, foo, quux}) => ..., {foo, quux, baz}). If I really wanted to change the ordering, I'd just do that in two places so the code reads consistently. Again, all of this is pretty minor but that's the point--the answer makes it seem like it's a big win and doesn't really explain that you could just as easily use multiple params, or that the last param is where the context args are passed into the callback, which is OP's fundamental problem.

[Dominik Matis]

Late to the party, but @chitzui, evalVar can be an array, so you can pass there multiple values

[ggorlen]

@DominikMatis No need for an array. You can just add more commas to the end of the callback parameter list and the evaluate function, e.g. page.evaluate((a, b, c) => ... , a, b, c). See this answer.

[nim118]

I am using soanrqube for static code analysis and when I use page.evaluate() and pass the variable described as above. I get this code smell - "'variable' is already declared in the upper scope."