IPIP-0524: Remove IPLD translation from IPFS HTTP Gateway

The Accept-header driven translation between various IPLD formats
is lossy, complicated, and often has us jumping through weird hoops
to massage data into an acceptable format. Maybe we can do without
it?

Simplifying the spec will make creating server implementations
easier and the translation can still be done by clients if they
require it.

Refs: ipfs/gateway-conformance#200

Author: achingbrain

src/http-gateways/path-gateway.md

@@ -158,12 +158,13 @@ For example:
 
 - [application/vnd.ipld.raw](https://www.iana.org/assignments/media-types/application/vnd.ipld.raw) – disables [IPLD codec deserialization](https://ipld.io/docs/codecs/), requests a verifiable raw [block](https://docs.ipfs.io/concepts/glossary/#block) to be returned
 - [application/vnd.ipld.car](https://www.iana.org/assignments/media-types/application/vnd.ipld.car) – disables [IPLD codec deserialization](https://ipld.io/docs/codecs/), requests a verifiable [CAR](https://docs.ipfs.io/concepts/glossary/#car) stream to be returned with implicit or explicit [`dag-scope`](https://specs.ipfs.tech/http-gateways/trustless-gateway/#dag-scope-request-query-parameter) for blocks at the terminus of the specified path and the blocks required to traverse path segments from root CID to the terminus.
-- [application/x-tar](https://en.wikipedia.org/wiki/Tar_(computing)) – returns UnixFS tree (files and directories) as a [TAR](https://en.wikipedia.org/wiki/Tar_(computing)) stream. Returned tree starts at a DAG which name is the same as the terminus segment. Produces 400 Bad Request for content that is not UnixFS.
-- [application/vnd.ipld.dag-json](https://www.iana.org/assignments/media-types/application/vnd.ipld.dag-json) – requests [IPLD Data Model](https://ipld.io/docs/data-model/) representation serialized into [DAG-JSON format](https://ipld.io/docs/codecs/known/dag-json/). If the requested CID already has `dag-json` (0x0129) codec, data is validated as DAG-JSON before being returned as-is. Invalid DAG-JSON produces HTTP Error 500.
-- [application/vnd.ipld.dag-cbor](https://www.iana.org/assignments/media-types/application/vnd.ipld.dag-cbor) – requests [IPLD Data Model](https://ipld.io/docs/data-model/) representation serialized into [DAG-CBOR format](https://ipld.io/docs/codecs/known/dag-cbor/). If the requested CID already has `dag-cbor` (0x71) codec,  data is validated as DAG-CBOR before being returned as-is. Invalid DAG-CBOR produces HTTP Error 500.
-- [application/json](https://www.iana.org/assignments/media-types/application/json) – same as `application/vnd.ipld.dag-json`, unless the CID's codec already is `json` (0x0200). Then, the raw JSON block can be returned as-is without any conversion.
-- [application/cbor](https://www.iana.org/assignments/media-types/application/cbor) – same as `application/vnd.ipld.dag-cbor`, unless the CID's codec already is `cbor` (0x51). Then, the raw CBOR block can be returned as-is without any conversion.
+- [application/x-tar](https://en.wikipedia.org/wiki/Tar_(computing)) – returns a UnixFS tree (files and directories) as a [TAR](https://en.wikipedia.org/wiki/Tar_(computing)) stream. Returned tree starts at a DAG which name is the same as the terminus segment. Produces 406 Not Acceptable for content that is not UnixFS.
+- [application/vnd.ipld.dag-json](https://www.iana.org/assignments/media-types/application/vnd.ipld.dag-json) – Validates block data as DAG-JSON before returning it as-is. Invalid DAG-JSON produces 406 Not Acceptable.
+- [application/vnd.ipld.dag-cbor](https://www.iana.org/assignments/media-types/application/vnd.ipld.dag-cbor) – Validates block data as DAG-CBOR before returning it as-is. Invalid DAG-CBOR produces 406 Not Acceptable.
+- [application/json](https://www.iana.org/assignments/media-types/application/json) – Validates block data as JSON before returning it as-is. Invalid JSON produces 406 Not Acceptable.
+- [application/cbor](https://www.iana.org/assignments/media-types/application/cbor) – Validates block data as CBOR before returning it as-is. Invalid CBOR produces 406 Not Acceptable.
 - [application/vnd.ipfs.ipns-record](https://www.iana.org/assignments/media-types/application/vnd.ipfs.ipns-record) – requests a verifiable :cite[ipns-record] to be returned. Produces 400 Bad Request if the content is not under the IPNS namespace, or contains a path.
+- [text/html](https://html.spec.whatwg.org/) – returns a human-readable representation of the requested data which may include a link to download the raw data.
 
 :::note
 

src/ipips/ipip-0524.md

@@ -0,0 +1,91 @@
+---
+title: "IPIP-0524: Remove IPLD Data Model translations from Gateways"
+date: 2025-11-27
+ipip: proposal
+editors:
+  - name: Alex Potsides
+    github: achingbrain
+    url: https://achingbrain.net
+    affiliation:
+        name: Shipyard
+        url: https://ipshipyard.com
+relatedIssues:
+  - https://github.com/ipfs/gateway-conformance/issues/200
+order: 524
+tags: ['ipips']
+---
+
+## Summary
+
+Make IPFS HTTP Gateway responses easier to reason about by not requiring IPLD
+Data Model translations
+
+## Motivation
+
+When sending an [Accept](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Accept)
+header or [format](https://specs.ipfs.tech/http-gateways/path-gateway/#format-request-query-parameter)
+query parameter to specify the response format of a request, the IPFS HTTP
+Gateway specs [allow translation](https://specs.ipfs.tech/http-gateways/path-gateway/#accept-request-header)
+of the requested content into the [IPLD Data Model](https://ipld.io/docs/data-model/).
+
+This adds significant complexity to HTTP Gateway implementations, since they
+need to be able to translate between arbitrary data types and handle all the
+various failure states.
+
+The conversions are also lossy due to differences in supported data types across
+different formats so lack general-purpose utility and are ultimately something
+that could be done on an interested client if required.
+
+## Detailed design
+
+An IPFS HTTP Gateway should make an attempt to verify that requested data can be
+represented in the desired response format (e.g. it should attempt to parse the
+block as JSON string or CBOR data when the relevant formats are requested,
+though in the case of CBOR it does not need to support all [tags](https://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml)
+encountered in the block).
+
+If the block was loaded successfully but cannot be represented in the requested
+format, the server should return a [406](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/406).
+
+This would be the outcome if, for example a DAG-JSON block was requested with
+the `application/cbor` format.
+
+Where a human-readable rendering of the data is desired, the `text/html` format
+can be requested. This would allow browsing DAG-PB data, for example.
+
+A [400](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/400)
+may be returned if the request was invalid (for example an unsupported format
+was requested).
+
+A [500](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/500)
+may be returned in other circumstances.
+
+## Design rationale
+
+Simplifying the HTTP Gateway spec to remove these format translations and the
+additional logic required makes it more straightforward to create new
+implementations, and makes the returned data more transparent and so easier to
+understand since the data is not modified to fit the output format.
+
+Clients that wish to translate between different data formats may request raw
+blocks and do the translation themselves.
+
+### User benefit
+
+HTTP Gateways will be simpler to use having fewer options and they will be
+encouraged to adapt their clients for their individual use-cases where more
+advanced functionality is required.
+
+### Compatibility
+
+This is a breaking change. Data will no longer be translated between different
+formats by the server and that logic will shift to interested clients.
+
+## Test fixtures
+
+Implementers can run the [gateway-conformance](https://github.com/ipfs/gateway-conformance/)
+test suite.
+
+### Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).