How to scrape "style" attribute value in Flutter/Dart

381

style attribute initially has no value. Value is set by JS while page rendering. You can try to scrape @data-bg instead:

imgUrls = imgUrl.map((element) => element.attributes['data-bg']).toList();
Share:
381
Faisal Shahzad
Author by

Faisal Shahzad

Updated on December 26, 2022

Comments

  • Faisal Shahzad
    Faisal Shahzad over 1 year

    I'm trying to scrape data from a webpage. I wanted to get the URL present inside the style attribute(as marked in picture)

    enter image description here

    So far I tried his but it returns me empty strings

    import 'dart:convert';
    import 'package:html/dom.dart' as dom;
    import 'package:html/parser.dart' as parser;
    import 'package:http/http.dart' as http;
    
    
    Future initiate() async {
      final response = await http.get('https://pixxelprecision.co.uk/portfolio');
      dom.Document document = parser.parse(response.body);
    
      final imgUrl = document.getElementsByClassName('w-ibanner-image');
      imgUrls = imgUrl.map((element) => element.attributes['style']).toList();
    
      print('URLs:  $imgUrls');
    
    }
    

    Console: enter image description here

  • Faisal Shahzad
    Faisal Shahzad over 3 years
    Thanks! that worked. But could you explain a little bit about 'data-bg' attribute cuz I can't see any attribute like this. Sorry I'm a noob so please don't mind my question. Thanks.
  • JaSON
    JaSON over 3 years
    @FaisalShahzad When you make investigations I guess you use DOM Inspector (F12) and you see elements and attributes of already rendered web-page. But note that when you make request with http-tool you're getting pure HTML source page (no JavaScript executed yet). To see this HTML you need to do Right click on web page in Browser -> "View Page Source". There you can find @data-bg which holds required url-value
  • Faisal Shahzad
    Faisal Shahzad about 3 years
    Hi, hope you are doing great. The solution you gave me is not working anymore. I have no idea what happened but now there is no "data-bg" attribute showing up in "View Page Source", and also the App give Null error because it can't find urls against 'data-bg'. I hope you had a solution for me this time also. Or is it possible to parse urls from the JavaScript loaded 'style' tag. So I don't have to worry again about this. Thanks.
  • JaSON
    JaSON about 3 years
    @FaisalShahzad yes, now @style moved directly to element HTML so your initial approach should work