OSError: [Errno 22] Invalid argument:

12,046

On Windows 10

I noticed the same behavior in my code and I found that I was using Microsoft OneDrive which was causing the same error. The file I was trying to open had its file pointer visible in Windows Explorer but not the contents. Are you using any cloud file sharing service?

(I right clicked the file, selected "Always Keep on this Device", ran the same code again and it worked).

Share:
12,046

Related videos on Youtube

Miguel Botelho
Author by

Miguel Botelho

Updated on June 04, 2022

Comments

  • Miguel Botelho
    Miguel Botelho almost 2 years

    I'm scraping a lot of reviews from a site with Python, for each review I call the "review" function and then open the file and append it to it. It works for a while but then the following error appears me everytime and not it the same review.

    OSError: [Errno 22] Invalid argument

    I tried json.dumps:

    scraped_data = reviews(line)
    with open('reviews','a' ) as f:
        f.write(json.dumps(scraped_data,f,indent = 4))
    

    but the same error keeps appearing. I also tried json.dump:

    scraped_data = reviews(line)
    with open('reviews','a' ) as f:
        json.dump(scraped_data,f,indent = 4))
    

    and, for some reason, I tried without indent too.

    edit: full traceback for json.dumps:

    Traceback (most recent call last):
    File "s.py", line 202, in <module>
    with open('reviews','a' ) as f:
    OSError: [Errno 22] Invalid argument: 'reviews' 
    

    full traceback for json.dump:

    Traceback (most recent call last):
    File "s.py", line 203, in <module>
    json.dump(scraped_data,f,indent = 4)
    OSError: [Errno 22] Invalid argument: 'reviews'
    
    • Mike
      Mike over 5 years
      Would you mind to share the full traceback?
    • Miguel Botelho
      Miguel Botelho over 5 years
      Yes, I edited the question
    • Mike
      Mike over 5 years
      I believe this would require further debugging, as it looks like, the problem is not with writing the dump to file but to dump the scraped data in the first place, try printing the scraped data to find out what's causing the problem.
    • Andrew Morozko
      Andrew Morozko over 5 years
      What's the exact Linux you are running on? The Errno 22 could have platform dependent meaning. I suggest running man 2 open, and reading descriptions for EINVAL messages. The best theory I came up with is O_APPEND ('a') is somehow badly supported on your system. Try using ('w'), this would overwrite file, but if this works then we know the problem is in 'a'.
    • Wodin
      Wodin over 5 years
      Possibly the OS telling you that you have too many files open? Do you get the same thing if you hard code scraped_data to a fixed string?
    • Andrew Morozko
      Andrew Morozko over 5 years
      Anoter idea just popped into my head: are you using multithreading or multiprocessing or viewing this file form another program? The root of all obscure bugs is concurrency ;)
    • Ulrich Eckhardt
      Ulrich Eckhardt over 5 years
      Does this really have anything to do with JSON or is this rather a problem opening files? Also, does it have to do anything with scraping at all, or does it just occur in that context? Please extract a minimal reproducible example from your code. Also, as a new user, please take the tour and read How to Ask.
  • Andrew Morozko
    Andrew Morozko over 5 years
    I tried, and at least on macOS and Debian 'a' works fine for creating new files...
  • SuperShoot
    SuperShoot over 5 years
    Can you explain how this will solve the OPs problem? Using a Context Manager to handle opening and closing files is ubiquitous.
  • Andrew Morozko
    Andrew Morozko over 5 years
    This would not change the call to open which produces an error, but would fail to close the file descriptor if something went wrong.
  • Andrew Morozko
    Andrew Morozko over 5 years
    :shrug: docs say that '+' is for reading and writing, but whatevs...
  • Sina Khelil
    Sina Khelil over 5 years
    You are correct @AndrewMorozko if it is not one of those then they are having an error that is either path or permissions related.
  • Andrew Morozko
    Andrew Morozko over 5 years
    yep, didn't thing of permissions, although there's EACCES for that. Can't we all just agree that errno mechanism is a great solution? Nothing confusing ever happens with it!
  • Miguel Botelho
    Miguel Botelho over 5 years
    I tried full path and 'a+' and it didn't work :( i'll try the above approach
  • Sina Khelil
    Sina Khelil over 5 years
    how did you build the full path? did you use os.path.join()? are you sure you have the correct permissions? make sure it is the user under which the app is running that has the permissions.
  • Sina Khelil
    Sina Khelil over 5 years
    If you are in windows, the ` \\ ` is an escape char and you cannot just use a string' you will need to properly escape it or use a raw string
  • Sina Khelil
    Sina Khelil over 5 years
    the \ not \\ is what I meant - SO made it difficult to rep in the comment