Web Browsing

PromptLoop helps you automatically research thousands of companies to find exactly what you need. Whether that is qualifying customers by market segment, monitoring their hiring page, checking what software they use, or analyzing their services. Get accurate data in minutes instead of hours of manual research.

How it works#

The web browsing tasks allow you to build AI systems that can extract specific data from websites. You define the item(s) you are looking for and then our models will handle the rest allowing you to input a website and get back a row of data corresponding to the search items you defined in the task. We then let you run these tasks on a CSV or Excel file with thousands of inputs (websites). Its as simple as defining what you are looking for and uploading a file.

Advanced options let you define exactly the format you need data in to effortlessly build a proprietary dataset from scratch in minutes.

Types of Tasks#

PromptLoop has three types of web browsing tasks:

Crawl Task - This takes a single website or domain and finds new datapoints (new columns) for it.

List Task - This takes a single website or domain and returns multiple rows with multiple datapoints for it. For example pulling out all companies from a customer page.

Search Task - This takes a search term and returns the matching link based on search criteria. This is a helpful starting point if you do not have a list of websites yet and need to find appropriate web resources.

Setting up#

You can copy a template or use the form to create a task for your data needs. You'll want an understanding of what the inputs will be (often a website or domain) and what the output columns you are looking for are. The form allows you to specify types for each item as well.

Running Tests#

You can immediately enter and input and run results right in your account. When you are satisfied and don't need further edits, you can upload your spreadsheet with a column that has the relevant inputs (whatever the task requires).

Viewing Data#

Once your dataset is run on the web task, you will be able to view all results right in the datasets tab and search, filter and save versions.

Tips and Tricks#

Designing an effective task depends on your end goal. This includes the tolerance for errors, importance of formatting, and how varying your input sites are.

• Focus - its best to set up tasks for specific purposes and only ask for the data you need.
• Formatting - PromptLoop Models will automatically and strictly format data for you, select the correct format for each output.
• Testing and Versions - It may take a few quick edits to get a task to work well. This is simple when using the editor and versions are always saved for future edits

Website Crawl Task#

Smart Crawling#

There are three different depths for the crawl tasks: Single page, Smart Crawl, and Deep Research Smart Crawl. This determines how far into the website our models will navigate, looking for answers to your search queries.

• Single Page will only look on the website page you provide as an input. For example, if we input www.promptloop.com/about into a single page search task then PromptLoop would only search the about page for information. This is useful if you don't want to navigate around a website and want to control where the information is extracted from. An example would be if you are looking at listing pages on a marketplace or listing website, single page ensures you don't get information from any other listings on the website.
• Smart Crawl is the default for all crawling tasks and uses the items you are looking for to intelligently navigate through the site, very similar to how a human would do it. This depth is sufficient for the majority of website crawl tasks and is optimized to allow enough navigation while also returning answers as quickly as possible.
• Deep Research Smart Crawl is a brand new crawling option we have released to our enterprise customers. This crawl type, enables our models to spend more time and explore more pages to make sure we are finding the answers to your queries wherever they may be on the website. Because we are doing significantly more crawling, this option will cause the task to run slightly slower.

Which should I use? For information that is located deep in a website, Deep research is the best option, but if the information you are looking for is always on the homepage, then basic Smart Crawl will be sufficient and run faster. We encourage customers to start with the Smart Crawl and if you notice that there are items not getting retrieved, then edit the task to use Deep Research. It takes no time to toggle between the two. If you only want information from a specific page, then Single Page is the best option for you.

Search Items#

Once a task is created, you will have the ability to tweak and edit instructions to optimize the results for your needs. Edits are broken up into the query used to find info an any additional instructions needed. In general we find these guidelines help our models extract relevant info.

Setting up effective search items is the start of most web extraction tasks. Whether you copy a template, start with the basic web task, or create your own, you will see an editor to select what items you are getting back.

Formatting#

Formatting allows you to collect data in exactly the format that you need. This includes pulling out website links, raw text, or numbers. You can also add additional instructions on top of formatting for more specific preferences.

Search Engine tricks#

Using web search engines instead of websites as a starting point for a task opens up a variety of powerful options. There are some nuances that can yield better results when selecting one or more links from a search to return.

Common Examples

• Find a company website from its name
• Find a linkedIn Profile of a specific person or title
• Find or summarize recent news articles about a topic or company

Query Options You can use standard search engine techniques within PromptLoop tasks. Here are a few that can work well.

site:name-of-site.com This will ensure results are ONLY from this specific site

-site:name-of-site.com This will EXCLUDE the site you include

apples AND oranges This will include results that appear for BOTH search terms limiting results

For help setting up advanced search tasks, get in touch for options to customize and optimize your results.

Why Leverage Our Web Browsing Capability#

Accuracy, Visibility, and Flexibility#

Our web browsing technology is engineered for speed. It allows you to navigate through multiple web pages in a fraction of the time it would take using traditional methods. This means you can quickly gather the data you need, making the most of your time and resources.

We run optimized resources to find the information you need directly from relevant sources. This allows for up-to-date, proprietary data delivered precisely in your required format. Because we leverage language models to navigate pages and identify relevant information, we can handle thousands of formats from millions of company page types, all at speeds unmatched by alternatives, including human-led research.

Unlike generative AI chat applications, PromptLoop uses models and techniques tuned to deliver accurate and formatted results only from trusted and provided sources. Without a source, we will not generate text for information, providing repeatable and reliable answers.

Scalable Solutions#

Whether you're looking to scrape data from a handful of web pages or perform Excel web scraping on a massive scale, our service is designed to scale with your needs. Our robust infrastructure ensures stability and performance, even when your data requirements grow exponentially.

We allow you to upload entire datasets to enrich using a CSV file or Excel file using the Datasets tool. This allows non-technical teams to create tasks to find precise information from company or entity websites and return formatted web scraping results for a bespoke dataset.

Precision and Customization#

Our web browsing feature is not just about accessing the web—it's about retrieving data with precision. With it, you can specify the data format you need, whether it's Booleans, direct answers to questions, or other specific data types that standard datasets may not provide.

Industry-Leading Web Scraping AI#

A cutting-edge web scraping AI is at the core of our web browsing capability. It navigates the web intelligently, understanding the context and semantics of the content to deliver relevant and accurate results.

Secure and Compliant#

We take web scraping seriously, adhering to the best practices and compliance standards. Our web browsing capability is optimized for you, allowing you to focus on the information and sources essential to driving your business decisions, not on setting up the specifics of a research pipeline.

For companies requiring an even higher level of data security than that which we offer standard to all customers, we can customize our infrastructure to meet your team's requirements. You can learn more by scheduling a call with our team today

Leveraging Web Browsing in Custom Tasks#

Our Custom Tasks utilize the full potential of our web browsing feature to cater to a wide range of business applications, such as:

• Company data web scraping: Extract detailed company profiles, financial data, or employee information.
• Market research: Gather insights on market trends, customer preferences, or competitive landscapes.
• Product analysis: Compare product features, prices, and availability across different e-commerce platforms.

With our web browsing capability, Custom Tasks can transform the web into a treasure trove of actionable data for your business.

Get Started#

Experience our web browsing feature's unparalleled efficiency and precision within Custom Tasks. Start harnessing the power of structured data extraction to make informed decisions and drive your business forward.