OSError: [Errno 22] Invalid argument:
On Windows 10
I noticed the same behavior in my code and I found that I was using Microsoft OneDrive which was causing the same error. The file I was trying to open had its file pointer visible in Windows Explorer but not the contents. Are you using any cloud file sharing service?
(I right clicked the file, selected "Always Keep on this Device", ran the same code again and it worked).
Related videos on Youtube
Miguel Botelho
Updated on June 04, 2022Comments
-
Miguel Botelho almost 2 years
I'm scraping a lot of reviews from a site with Python, for each review I call the "review" function and then open the file and append it to it. It works for a while but then the following error appears me everytime and not it the same review.
OSError: [Errno 22] Invalid argument
I tried json.dumps:
scraped_data = reviews(line) with open('reviews','a' ) as f: f.write(json.dumps(scraped_data,f,indent = 4))
but the same error keeps appearing. I also tried json.dump:
scraped_data = reviews(line) with open('reviews','a' ) as f: json.dump(scraped_data,f,indent = 4))
and, for some reason, I tried without indent too.
edit: full traceback for json.dumps:
Traceback (most recent call last): File "s.py", line 202, in <module> with open('reviews','a' ) as f: OSError: [Errno 22] Invalid argument: 'reviews'
full traceback for json.dump:
Traceback (most recent call last): File "s.py", line 203, in <module> json.dump(scraped_data,f,indent = 4) OSError: [Errno 22] Invalid argument: 'reviews'
-
Mike over 5 yearsWould you mind to share the full traceback?
-
Miguel Botelho over 5 yearsYes, I edited the question
-
Mike over 5 yearsI believe this would require further debugging, as it looks like, the problem is not with writing the dump to file but to dump the scraped data in the first place, try printing the scraped data to find out what's causing the problem.
-
Andrew Morozko over 5 yearsWhat's the exact Linux you are running on? The Errno 22 could have platform dependent meaning. I suggest running
man 2 open
, and reading descriptions forEINVAL
messages. The best theory I came up with isO_APPEND
('a'
) is somehow badly supported on your system. Try using ('w'
), this would overwrite file, but if this works then we know the problem is in'a'
. -
Wodin over 5 yearsPossibly the OS telling you that you have too many files open? Do you get the same thing if you hard code
scraped_data
to a fixed string? -
Andrew Morozko over 5 yearsAnoter idea just popped into my head: are you using multithreading or multiprocessing or viewing this file form another program? The root of all obscure bugs is concurrency ;)
-
Ulrich Eckhardt over 5 yearsDoes this really have anything to do with JSON or is this rather a problem opening files? Also, does it have to do anything with scraping at all, or does it just occur in that context? Please extract a minimal reproducible example from your code. Also, as a new user, please take the tour and read How to Ask.
-
-
Andrew Morozko over 5 yearsI tried, and at least on macOS and Debian 'a' works fine for creating new files...
-
SuperShoot over 5 yearsCan you explain how this will solve the OPs problem? Using a Context Manager to handle opening and closing files is ubiquitous.
-
Andrew Morozko over 5 yearsThis would not change the call to
open
which produces an error, but would fail to close the file descriptor if something went wrong. -
Andrew Morozko over 5 years:shrug: docs say that '+' is for reading and writing, but whatevs...
-
Sina Khelil over 5 yearsYou are correct @AndrewMorozko if it is not one of those then they are having an error that is either path or permissions related.
-
Andrew Morozko over 5 yearsyep, didn't thing of permissions, although there's
EACCES
for that. Can't we all just agree that errno mechanism is a great solution? Nothing confusing ever happens with it! -
Miguel Botelho over 5 yearsI tried full path and 'a+' and it didn't work :( i'll try the above approach
-
Sina Khelil over 5 yearshow did you build the full path? did you use
os.path.join()
? are you sure you have the correct permissions? make sure it is the user under which the app is running that has the permissions. -
Sina Khelil over 5 yearsIf you are in windows, the ` \\ ` is an escape char and you cannot just use a string' you will need to properly escape it or use a raw string
-
Sina Khelil over 5 yearsthe \ not \\ is what I meant - SO made it difficult to rep in the comment