How to scrape "style" attribute value in Flutter/Dart
381
style
attribute initially has no value. Value is set by JS while page rendering. You can try to scrape @data-bg
instead:
imgUrls = imgUrl.map((element) => element.attributes['data-bg']).toList();
Author by
Faisal Shahzad
Updated on December 26, 2022Comments
-
Faisal Shahzad over 1 year
I'm trying to scrape data from a webpage. I wanted to get the URL present inside the style attribute(as marked in picture)
So far I tried his but it returns me empty strings
import 'dart:convert'; import 'package:html/dom.dart' as dom; import 'package:html/parser.dart' as parser; import 'package:http/http.dart' as http; Future initiate() async { final response = await http.get('https://pixxelprecision.co.uk/portfolio'); dom.Document document = parser.parse(response.body); final imgUrl = document.getElementsByClassName('w-ibanner-image'); imgUrls = imgUrl.map((element) => element.attributes['style']).toList(); print('URLs: $imgUrls'); }
-
Faisal Shahzad over 3 yearsThanks! that worked. But could you explain a little bit about 'data-bg' attribute cuz I can't see any attribute like this. Sorry I'm a noob so please don't mind my question. Thanks.
-
JaSON over 3 years@FaisalShahzad When you make investigations I guess you use DOM Inspector (F12) and you see elements and attributes of already rendered web-page. But note that when you make request with
http
-tool you're getting pure HTML source page (no JavaScript executed yet). To see this HTML you need to do Right click on web page in Browser -> "View Page Source". There you can find@data-bg
which holds required url-value -
Faisal Shahzad about 3 yearsHi, hope you are doing great. The solution you gave me is not working anymore. I have no idea what happened but now there is no "data-bg" attribute showing up in "View Page Source", and also the App give Null error because it can't find urls against 'data-bg'. I hope you had a solution for me this time also. Or is it possible to parse urls from the JavaScript loaded 'style' tag. So I don't have to worry again about this. Thanks.
-
JaSON about 3 years@FaisalShahzad yes, now
@style
moved directly to element HTML so your initial approach should work