You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Error reading png image from URL with imread in matplotlib 3.3 #18129
Just looked at the source. urllib.request is used and passes the response to PIL.PngImagePlugin.PngImageFile, but if it passes it to PIL.Image.open, it would work.
Edit:
The response object is not seekable which causes the error. You can wrap it in BytesIO to make it work.
PIL.Image.open supports urllib.request.urlopen objects (python-pillow/Pillow#1151) so I don't see why we can't just always pass it there instead of PngImagePlugin.PngImageFile
The point was to force reading the file as a png (if ext == "png") rather than relying on pillow's autodetection. This only matters if someone is nasty enough to have e.g. a jpeg file in a png extension, or pass a jpeg file to imread(..., ext="png").
My guess is that wrapping in a BytesIO() is fine, you already made a web request anyways and we're about to convert the whole thing to float32, so I wouldn't worry too much about speed or memory usage.
Labeling this as a good first issue because there is a very clear regression and 2 suggested fixes in this thread. This should get a test (so we don't accidentally break it again). I think we have a test of getting a stylesheet from a URL, the new test should get the same markup so it can be skipped in no-network environments (ex debian builds) .
https://docs.python.org/3.8/library/urllib.request.html#urllib.request.urlopen
For HTTP and HTTPS URLs, this function returns a http.client.HTTPResponse object slightly modified. In addition to the three new methods above, the msg attribute contains the same information as the reason attribute — the reason phrase returned by server — instead of the response headers as it is specified in the documentation for HTTPResponse.
For FTP, file, and data URLs and requests explicitly handled by legacy URLopener and FancyURLopener classes, this function returns a urllib.response.addinfourl object.
Do we expect that type instability from urlopen? If not, does this need to be reported as a bug to cpython?
HTTPResponse is documented to implement BufferedIOBase, which does not include seek(). IMHO that's all we can expect. That file:// returns another object with more functionality can be misleading but is not strictly a bug and I don't think cpython could/would change this.
Like Pillow ((python-pillow/Pillow#1151) we cannot assume that seek() exists. The fix is to either pass the response straight through (which we apparently do not want to do - did not follow the details), or apply the same fix with wrapping if neccesary as Pillow does.
@linchiwei123@andrewkoo I went ahead and opened the PR to take care of this. Sorry to steal the work out from under you but as this was the last issue for 3.3.1 we wanted to expedite the process.
It's the latest version. And I don't think update was the fix earlier. I searched it everywhere and it still does not have proper solution. If you can help me or refer some fixes it will be of great help.
The issue is as shown below
Please open a new issue with a self-contained example (ie real url). Howver note that the original issue above you pass the url as a string not whatever urlopen returns.
Activity
jklymak commentedon Jul 30, 2020
We switched to pillow handling the image i/o and it seems pillow can't handle urls? #15193
tdpetrou commentedon Jul 30, 2020
Would a simple approach using the standard library's
urllib.requestmodule to read the contents as bytes not work?tdpetrou commentedon Jul 30, 2020
Just looked at the source.
urllib.requestis used and passes the response toPIL.PngImagePlugin.PngImageFile, but if it passes it toPIL.Image.open, it would work.Edit:
The response object is not seekable which causes the error. You can wrap it in
BytesIOto make it work.This is essentially the error within urllib
tdpetrou commentedon Jul 30, 2020
PIL.Image.opensupportsurllib.request.urlopenobjects (python-pillow/Pillow#1151) so I don't see why we can't just always pass it there instead ofPngImagePlugin.PngImageFilejklymak commentedon Jul 30, 2020
This was discussed - #15193 (comment) Not clear to me what @anntzer was trying to do with that, so perhaps he can explain.
anntzer commentedon Jul 30, 2020
The point was to force reading the file as a png (if
ext == "png") rather than relying on pillow's autodetection. This only matters if someone is nasty enough to have e.g. a jpeg file in a png extension, or pass a jpeg file toimread(..., ext="png").My guess is that wrapping in a BytesIO() is fine, you already made a web request anyways and we're about to convert the whole thing to float32, so I wouldn't worry too much about speed or memory usage.
tacaswell commentedon Aug 1, 2020
Labeling this as a good first issue because there is a very clear regression and 2 suggested fixes in this thread. This should get a test (so we don't accidentally break it again). I think we have a test of getting a stylesheet from a URL, the new test should get the same markup so it can be skipped in no-network environments (ex debian builds) .
andrewkoo commentedon Aug 1, 2020
Hi! I am new to open source and am interested in helping out if I can. If this is still open, can I start working on this?
edit: Looking at tests/test_image.py it looks like there is a test_load_from_url() already defined:
Does this test case not catch this issue?
tacaswell commentedon Aug 3, 2020
@andrewkoo Interesting, it is probably worth investigating why that test did not catch this issue!
timhoffm commentedon Aug 3, 2020
Without checking, I'd guess file:/// vs. https:///.
dopplershift commentedon Aug 3, 2020
With checking:
open()doesn't seem to recognizefile:///, but it's possible I did something wrong.linchiwei123 commentedon Aug 3, 2020
need to test http(s) as well.
1 remaining item
linchiwei123 commentedon Aug 4, 2020
https://docs.python.org/3.8/library/urllib.request.html#urllib.request.urlopen
For HTTP and HTTPS URLs, this function returns a http.client.HTTPResponse object slightly modified. In addition to the three new methods above, the msg attribute contains the same information as the reason attribute — the reason phrase returned by server — instead of the response headers as it is specified in the documentation for HTTPResponse.
For FTP, file, and data URLs and requests explicitly handled by legacy URLopener and FancyURLopener classes, this function returns a urllib.response.addinfourl object.
timhoffm commentedon Aug 4, 2020
HTTPResponse is documented to implement BufferedIOBase, which does not include
seek(). IMHO that's all we can expect. That file:// returns another object with more functionality can be misleading but is not strictly a bug and I don't think cpython could/would change this.Like Pillow ((python-pillow/Pillow#1151) we cannot assume that
seek()exists. The fix is to either pass the response straight through (which we apparently do not want to do - did not follow the details), or apply the same fix with wrapping if neccesary as Pillow does.FIX: fix reading from http/https urls via imread
tacaswell commentedon Aug 5, 2020
@linchiwei123 @andrewkoo I went ahead and opened the PR to take care of this. Sorry to steal the work out from under you but as this was the last issue for 3.3.1 we wanted to expedite the process.
andrewkoo commentedon Aug 5, 2020
All good! This is all just a learning process for me so would hate to delay anything..
linchiwei123 commentedon Aug 5, 2020
It's OK.
FIX: fix reading from http/https urls via imread
FIX: fix reading from http/https urls via imread
anntzer commentedon Oct 16, 2020
Note that with Pillow 8.0 the dance at #18129 (comment) would not be necessary anymore per https://pillow.readthedocs.io/en/stable/releasenotes/8.0.0.html#image-open-add-formats-parameter.
gaurav6386 commentedon Oct 25, 2021
What is the fix guys?
jklymak commentedon Oct 25, 2021
Upgrade to 3.3.1? Or are you saying that newer matplotlib don't work?
gaurav6386 commentedon Oct 25, 2021
It's the latest version. And I don't think update was the fix earlier. I searched it everywhere and it still does not have proper solution. If you can help me or refer some fixes it will be of great help.
The issue is as shown below
But it exits after throwing
jklymak commentedon Oct 25, 2021
Please open a new issue with a self-contained example (ie real url). Howver note that the original issue above you pass the url as a string not whatever urlopen returns.