How to Scrape the Content of a URL / Website Page Behind a Paywall

If you need to get access to an article behind a paywall in order to analyze it with InfraNodus, you can do so using the built-in URL import app or the Notion Web Clipper extension. 

Once you have the text of the article, you can import it to InfraNodus and visualize it as a graph to detect the main topics in the article as well as the structural gaps and identify the content that's missing.

In order to do that:

1. Install the Notion Web Clipper Plugin or use the URL import

Use InfraNodus URL import app or install the Notion Web Clipper extension into your browser.

While both can get behind a paywall, Notion is a multi-million company, so their web clipper is particularly good for getting behind the paywalls. Go to the article that you'd like to read, and click the Notion extension button. Save the article, and then go to Notion to open its full text.

2. Copy and Paste the Article to InfraNodus

Then copy the text of the article into your clipboard, go to InfraNodus.Com and use the Add a New Graph app to paste the text to analyze:

You will see the main topics and concepts in that text, the topical clusters, and the relations between them.