Skip to content

Instantly share code, notes, and snippets.

@koyachi

koyachi/scrape.dart

Forked from kui/scrape.dart
Last active April 7, 2018 01:09
Show Gist options
  • Save koyachi/d802c607223888651b5719a38fd9e4d8 to your computer and use it in GitHub Desktop.
Save koyachi/d802c607223888651b5719a38fd9e4d8 to your computer and use it in GitHub Desktop.
Download ZIP
a web scraping script with Dart and html
Raw
scrape.dart
import 'dart:io';
import 'dart:async';
import 'package:html5lib/parser.dart';
import 'package:html5lib/dom.dart';
main() {
final url = 'http://comic-walker.com/';
getHtml(url, (document) {
// page title
print(document.querySelector('title').text);
// Newer comics
document.querySelectorAll('#bookList > li').forEach((e) {
print(e.querySelector('.list_bookName').text);
});
});
}
/// fetch the HTML from [url] then execute [f] with the parsed HTML
Future getHtml(String url, f(Document docment)) =>
new HttpClient()
.getUrl(Uri.parse(url))
.then((req) => req.close())
.then((res) => res
.asyncExpand((bytes) => new Stream.fromIterable(bytes))
.toList())
.then((bytes) => parse(bytes, sourceUrl: url))
.then(f);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment