Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Changelog

## 0.1.0 - 2026-11-17

Plugin version stays 0.1.0. It is not on the Dify Marketplace.

### Changed

- Pin `diffbot==3.4.0` in `diffbot/requirements.txt` and `diffbot/pyproject.toml`.
- Remove the editable `/workspace/diffbot-python` path from `requirements-dev.txt`.
- CI installs the pin from PyPI and runs the SDK-dependent tests (the skip in `tests/conftest.py` is gone).

This repo does not publish to npm or PyPI. Shipping is a Dify Marketplace pull request, which is separate from merging this branch. Feature-detect fallbacks in `diffbot/tools/sdk.py` are still in the tree for that submission.
22 changes: 15 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,26 +14,34 @@ The installable plugin is in [`diffbot/`](diffbot/). Its [README](diffbot/README
| --- | --- |
| Web search | `GET https://llm.diffbot.com/api/v1/web_search` (bearer token) |
| Extract page | `GET https://api.diffbot.com/v3/{analyze\|article\|product\|discussion\|image\|video\|list\|event}` (`token` query parameter) |
| Search organizations | `GET https://kg.diffbot.com/kg/v3/dql` |
| Search organizations | `POST https://kg.diffbot.com/kg/v3/dql` |
| Search people | same DQL endpoint, public professional profiles only |
| Search news | same DQL endpoint, newest publication date first via `sortBy:date` |
| Enhance entity | `GET https://kg.diffbot.com/kg/v3/enhance` |
| Start crawl | `GET https://api.diffbot.com/v3/crawl` (create only; returns a job name) |
| Crawl jobs | list / status / results / **delete (destructive)** on `/v3/crawl` and `/v3/crawl/data` |
| Resolve entities | `POST https://nl.diffbot.com/v1/` |

Saving the credential checks `GET https://api.diffbot.com/v4/account?days=1`. Get a free token at [app.diffbot.com/get-started](https://app.diffbot.com/get-started).

Crawl, raw DQL, ontology lookup, places, and deals are not in v0.1. See SUBMISSION.md for why.
Raw DQL, ontology lookup, Ask, places, and deals are not in this version. See SUBMISSION.md for why.

All HTTP goes through the [Diffbot Python SDK](https://github.com/diffbot/diffbot-python) (`diffbot.Diffbot`), behind a transport that refuses private, loopback, and cloud-metadata hosts.

## Run the tests

Python 3.12:
Python 3.12+:

```bash
pip install -r diffbot/requirements.txt pytest
pytest
python -m venv .venv && . .venv/bin/activate
pip install -r requirements-dev.txt # dify_plugin, pytest, and diffbot==3.4.0
pytest -q
```

The tests mock HTTP and make no Diffbot calls. `diffbot/requirements.txt` pins `dify_plugin==0.11.0` and `requests==2.34.2`.
`diffbot/requirements.txt` pins `diffbot==3.4.0`, which is on PyPI. `requirements-dev.txt` does not contain a machine-local path.

The tests serve every request from `httpx.MockTransport` and make no Diffbot calls. The wiring tests run against the SDK as installed and against a copy with the 3.3.0 methods removed, which covers the plugin's `client.request()` fallbacks (`verify_token`, `crawl_download`, extract options).

GitHub CI (`.github/workflows/test.yml`) installs `diffbot/requirements.txt`, including `diffbot==3.4.0`, and runs the full suite.

## Package

Expand Down
197 changes: 124 additions & 73 deletions SUBMISSION.md

Large diffs are not rendered by default.

3 changes: 1 addition & 2 deletions diffbot/PRIVACY.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,9 @@ Every tool call leaves the Dify workspace and is processed by Diffbot Technologi
| Search query, result count, snippet size | `llm.diffbot.com` Web Search | Run a web search. The token is sent as a bearer header. |
| Page URL and page type | `api.diffbot.com` Extract | Fetch and structure one public page. Diffbot retrieves the URL. |
| Names, homepages, categories, places, employee and revenue bounds, article phrases, publishers, dates | `kg.diffbot.com` DQL | Search the Knowledge Graph. |
| Person or organization name, public URL, location text, employer, title | `kg.diffbot.com` Enhance | Match a Knowledge Graph record. |
| Plain text from the user, up to 20,000 characters, and an optional language code | `nl.diffbot.com` | Recognize and link named entities. |

The plugin does not offer parameters for email address, phone number, or IP address, including on Enhance, where the Diffbot API supports those fields. If a Knowledge Graph response still contains email or phone values, the plugin drops them before returning the tool result. It also drops article HTML and, unless the news tool is asked for an excerpt, article body text.
The plugin does not offer parameters for email address, phone number, or IP address. If a Knowledge Graph response still contains email or phone values, the plugin drops them before returning the tool result. It also drops article HTML and, unless the news tool is asked for an excerpt, article body text.

People and organization results can still contain personal data that is already public on the web: names, job titles, employers, cities, countries, homepage URLs, and LinkedIn profile URLs. Diffbot describes people in its index as search subjects whose information was published on publicly available sites. News results can quote public articles. Entity resolution can identify people named in text the user pasted.

Expand Down
16 changes: 9 additions & 7 deletions diffbot/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Search the public web and the Diffbot Knowledge Graph from a Dify agent or workflow.

Diffbot structures pages and keeps a knowledge graph of organizations, people, and articles. This plugin exposes a small v1 of that API: web search, page extraction, organization and people lookup, news search, entity enhancement, and named-entity resolution. A free token from [app.diffbot.com/get-started](https://app.diffbot.com/get-started) is enough to authenticate.
Diffbot structures pages and keeps a knowledge graph of organizations, people, and articles. This plugin exposes a small v1 of that API: web search, page extraction, organization and people lookup, news search, and named-entity resolution. A free token from [app.diffbot.com/get-started](https://app.diffbot.com/get-started) is enough to authenticate.

Published by Diffbot. Source: https://github.com/diffbot/dify-diffbot. Support: support@diffbot.com.

Expand All @@ -12,29 +12,31 @@ Published by Diffbot. Source: https://github.com/diffbot/dify-diffbot. Support:
| --- | --- | --- |
| Web search | `GET https://llm.diffbot.com/api/v1/web_search` | Ranked pages with a score, URL, date, and passage. A query that is only `url:https://...` reads the indexed copy. |
| Extract page | `GET https://api.diffbot.com/v3/analyze` or a typed extract path | Live extraction of one public page. `article` is the usual choice for news and blogs. |
| Search organizations | `GET https://kg.diffbot.com/kg/v3/dql` | Companies by name, homepage, category, headquarters, headcount, revenue, or public status. |
| Search organizations | `POST https://kg.diffbot.com/kg/v3/dql` | Companies by name, homepage, category, headquarters, headcount, revenue, or public status. |
| Search people | same DQL endpoint | Public professional profiles by name, employer, title, or location. |
| Search news | same DQL endpoint | Articles by mentioned entity, phrase, publisher, language, or date. |
| Enhance entity | `GET https://kg.diffbot.com/kg/v3/enhance` | Best Knowledge Graph match for a person or organization from a name, URL, or employer. |
| Start crawl | `GET https://api.diffbot.com/v3/crawl` | Start a background crawl of public seed URLs. Returns a job name, never results. |
| Crawl jobs | `/v3/crawl`, `/v3/crawl/data` | List jobs, check status, fetch results (up to 1000 rows, truncated), or delete a job. **Delete is destructive** and removes the job and its data. |
| Resolve entities | `POST https://nl.diffbot.com/v1/` | Names in a passage, linked to Knowledge Graph ids when Diffbot can match them. |

Crawl, raw DQL, ontology lookup, places, and deals are not in this version. Industry filters must already be exact category names such as `Software Companies`. The plugin does not resolve free text onto the taxonomy.
Raw DQL, ontology lookup, Ask, places, and deals are not in this version. Industry filters must already be exact category names such as `Software Companies`. The plugin does not resolve free text onto the taxonomy.

## Credentials

1. Create a token at [app.diffbot.com/get-started](https://app.diffbot.com/get-started).
2. In Dify, open Plugins, install this package, and choose Diffbot.
3. Paste the token into **Diffbot token**.
4. Optional: set **Request timeout** (5 to 300 seconds, default 30).

Dify stores the value as a secret. The plugin sends it only to the Diffbot hosts listed in `manifest.yaml`. Web Search uses `Authorization: Bearer`. The other calls use the `token` query parameter, which is how those APIs authenticate. Saving the credential calls `GET https://api.diffbot.com/v4/account` with `days=1` and does not show the account payload.

The token needs access to Extract, Web Search, Knowledge Graph, Enhance, and Natural Language. A free plan may not include every product. If a call returns an authorization error after the account check succeeds, confirm that product is enabled for the token.
The token needs access to Extract, Web Search, Knowledge Graph, and Natural Language. A free plan may not include every product. If a call returns an authorization error after the account check succeeds, confirm that product is enabled for the token.

## Use the tools

In an Agent, add the Diffbot tools and ask for a company, a news topic, or a URL. In a Workflow, add a tool node and pick one action. Required fields are marked on each tool. Limits are small on purpose: at most 10 search rows, 3 enhance matches, and 20,000 characters of text for entity resolution.
In an Agent, add the Diffbot tools and ask for a company, a news topic, or a URL. In a Workflow, add a tool node and pick one action. Required fields are marked on each tool. Limits: up to 50 web results, 100 Knowledge Graph rows per call (page with Offset), and 20,000 characters of text for entity resolution. Extract has optional caller-set options (fields, timeout, mode, paging, natural language, and others).

Extract accepts only `http` and `https` URLs. Localhost, link-local, and private literal addresses are rejected before the request is sent.
Extract accepts `http` and `https` URLs; a bare domain such as `example.com` gets `https://`. Localhost, link-local, and private literal addresses are rejected before the request is sent.

## Install the package locally

Expand Down
9 changes: 7 additions & 2 deletions diffbot/provider/diffbot.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,18 @@
from dify_plugin.errors.tool import ToolProviderCredentialValidationError

from tools.client import DiffbotError
from tools.common import make_client
from tools.sdk import make_client, mapped_errors, verify_token


class DiffbotProvider(ToolProvider):
def _validate_credentials(self, credentials: dict[str, Any]) -> None:
try:
make_client(credentials).validate_token()
client = make_client(credentials)
with client, mapped_errors(client):
# verify_token() never hands back the account payload (it contains the token).
valid = verify_token(client)
if not valid:
raise DiffbotError("Invalid or unauthorized Diffbot token.")
except DiffbotError as exc:
raise ToolProviderCredentialValidationError(str(exc)) from exc
except Exception as exc:
Expand Down
13 changes: 12 additions & 1 deletion diffbot/provider/diffbot.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,24 @@ credentials_for_provider:
help:
en_US: A free token is available at app.diffbot.com/get-started. Dify stores it as a secret and the plugin sends it only to Diffbot.
url: https://app.diffbot.com/get-started
request_timeout:
type: text-input
required: false
default: "30"
label:
en_US: Request timeout (seconds)
placeholder:
en_US: "30"
help:
en_US: Seconds to wait for each Diffbot call, from 5 to 300. Leave blank for 30.
tools:
- tools/search_web.yaml
- tools/extract.yaml
- tools/search_organizations.yaml
- tools/search_people.yaml
- tools/search_news.yaml
- tools/enhance_entity.yaml
- tools/crawl_start.yaml
- tools/crawl_jobs.yaml
- tools/resolve_entities.yaml
extra:
python:
Expand Down
2 changes: 1 addition & 1 deletion diffbot/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,5 @@ readme = "README.md"
requires-python = ">=3.12"
dependencies = [
"dify_plugin==0.11.0",
"requests==2.34.2",
"diffbot==3.4.0",
]
2 changes: 1 addition & 1 deletion diffbot/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
dify_plugin==0.11.0
requests==2.34.2
diffbot==3.4.0
Loading
Loading