WebSonar

DEGA Partners is a law firm with a brand-protection practice. When somebody puts a brand on something they have no right to, the lawyers there build a violation case and send it. In 2019 they ordered a tool from me to take the manual part of that job off their hands, and that tool became WebSonar.

The manual part was the collecting. A case about the illegal use of a brand needs every relevant page of the offending website, and before WebSonar the lawyers went and got those pages by hand. One website, one page at a time, side by side with the case they were putting together, and the pages kept changing while they did it. That works for a couple of sites. It doesn't really work when you want the practice to take on more, and more was pretty much the whole point: help the lawyers work efficiently and at a bigger scale.

WebSonar ran as a local web application that the lawyer opened in a browser, with 4 screens: the tasks, the schedule, the queue and the reports. Each task had its own keyword list, and then WebSonar went looking. It found sites through the Google Custom Search, and it also searched two Ukrainian marketplaces, each one through its own search engine. A lawyer could keep an exclusion list for the sites they never wanted to see again, or switch discovery off completely and check only the sites they named, with a page limit on each one.

On every page it read, WebSonar matched the task's keywords. A page with no match got dropped and left no trace. A page with a match got kept twice: as an image of the page as it rendered, and as an image of the page source, with the matched words highlighted in both.

Two images, because they answer two different questions. The rendered one shows what a visitor saw. The source shows the matched words inside the markup, including the ones a site keeps out of a visitor's sight. And a violation often gets acted on months after it's found, when the page has been edited or taken down already, so the lawyer opening the case sees the page the way it was, with the match marked right in it.

While it read a page, WebSonar also went through every link on it. Phone and email links went into the site's contact list, links to social networks were collected on their own, and everything else became the list of pages still to check. For each site it also read the domain registration record, the WHOIS, in the same headless browser as everything else.

All of that ended up in PDF reports, one overall report for each task and one for each site. The overall report counted the sites found and every keyword match across them. The site report held the contacts and the WHOIS data, the whole contact trail for that site. The headless browser printed the PDFs itself, so there was no PDF library in the product at all. If a report file went missing, WebSonar rebuilt it on request. From there the lawyers took over, built the violation case and sent it.

flowchart LR P[Page in a headless browser] --> M[Keyword match] M --> E[Rendered image and page source, both highlighted] E --> C[Site contacts and domain record] C --> R[One PDF per site, one overall] R --> S[Lawyer builds and sends the case]

The rest was about letting the lawyers leave it running. A scheduler put a saved task into the queue on a chosen date, or weekly on chosen weekdays at a chosen time. The queue showed when each task was queued, how long a run had been going and when it expected to finish, and it had a stop control that worked in the middle of a run, because WebSonar checked the queue before every site and every page. Runs could resume, too: the found site list was saved into the task before scanning started, so a re-run went straight past discovery to the actual work. It went through the sites one at a time and through the pages one at a time.

Under the hood it was Node.js with Express, Puppeteer driving headless Chromium for every capture, every report and the WHOIS read, the Google Custom Search API, and a flat JSON file holding the tasks, the schedule, the queue and the history. The frontend was jQuery.

WebSonar was my own product. DEGA Partners ordered it and licensed it. The big runs started in late 2019, and the system has worked ever since. In 2024 it got an upgrade with much better numbers: 10 browsers capturing at once, and 2,010 pages done in under 28 minutes. The lawyers still use it.