I have deeply mixed feelings about #ActivityPub's adoption of JSON-LD, as someone who's spent way too long dealing with it while building #Fedify.
-
@sl007 @hongminhee @julian i feel like you're falling into a trap i've seen a lot around AP spaces: just because the data can be contorted to represent something does not mean software will interpret it as such.
any software who wants to support wikidata statements and relations will have to go out of their way to implement that manually with or without json-ld in the mix, and interoperability between those software will have to specify how that works. and in your specification you can indeed make it so Simply Linking to the wikidata json-ld (which i don't believe it provides out of the box, it does for xml and n-triples, if we're talking about rdf. if not, their bespoke json format is just as authoritative) can work (but i'd say using the Qxxx and Pxx IDs and letting the software figure out how to access it would be better!)
if you have the dream of making an as:Note and having it's as:attributedTo be the wikidata entity for alan turing... sorry, nobody other than maybe your own software will even attempt interpreting that@hongminhee @sl007 @julian attempting to support this kind of "data contortion" (i made this up and prolly isnt the right way to describe this) would rapidly balloon the scope of every fedi software ever. i don't believe anyone would want to develop for such ecosystem
a similar example i saw was someone attempting to explain how you can partially inline an as:object you as:Like'd in order to specify you only liked that past version of it and if it changed your like shouldn't count. without describing this exact scenario i don't believe any software, json-ld capable or not, would interpret that Like as such. same thing with the long-form text FEP which attempts to support non-activitypub authors -
@hongminhee @sl007 @julian attempting to support this kind of "data contortion" (i made this up and prolly isnt the right way to describe this) would rapidly balloon the scope of every fedi software ever. i don't believe anyone would want to develop for such ecosystem
a similar example i saw was someone attempting to explain how you can partially inline an as:object you as:Like'd in order to specify you only liked that past version of it and if it changed your like shouldn't count. without describing this exact scenario i don't believe any software, json-ld capable or not, would interpret that Like as such. same thing with the long-form text FEP which attempts to support non-activitypub authorsit is just damned simple, your as: Client can do so much by asking wikidata, OSM, federated geocoding and not our system. When you use a property for the first time, the client can cache its names in the languages of the user etc.
-
it is just damned simple, your as: Client can do so much by asking wikidata, OSM, federated geocoding and not our system. When you use a property for the first time, the client can cache its names in the languages of the user etc.
@sl007 @hongminhee @julian i genuinely can't see where json-ld is relevant here. if your client wants to support wikidata and OSM then it can do that with or without json-ld being involved. you are going to have to document how this integration works anyhow if you want anyone else to do so -
@hongminhee How hard would it be for a future version of ActivityPub to simply back out JSON-LD support? Would there be a downside to this?
-
@hongminhee How hard would it be for a future version of ActivityPub to simply back out JSON-LD support? Would there be a downside to this?
@mcc@mastodon.social I'm not sure, but that would break some ActivityPub implementations relying on JSON-LD processors.
-
@hongminhee if i can give one piece of advice to devs who want to process JSON-LD: dont bother compacting. you already know the schema you output (or you're just passing through what the user gives and it doesn't matter to you), serialize directly to the compacted representation, and only run expansion on incoming data
expansion is the cheapest JSON-LD operation (since all other operations depend on it and run it internally anyhow), and this will get you all the compatibility benefits of JSON-LD with little downsides (beyond more annoying deserialization code, as you have to map the expanded representation to your internal structure which will likely be modeled after the compacted one)generally agreed except
> you have to map the expanded representation to your internal structure which will likely be modeled after the compacted one
this is compaction but manual instead of using a jsonld processor to do it. maybe the more precise argument is "don't bother with auto/native compaction"?
with that said: you also lose out on flattening and framing, which are pretty cool features for transforming the serialization. if you don't care about those, ok fine
-
generally agreed except
> you have to map the expanded representation to your internal structure which will likely be modeled after the compacted one
this is compaction but manual instead of using a jsonld processor to do it. maybe the more precise argument is "don't bother with auto/native compaction"?
with that said: you also lose out on flattening and framing, which are pretty cool features for transforming the serialization. if you don't care about those, ok fine
@trwnh @hongminhee i'm not entirely sure on what you mean (it's about 3am here) but compaction isnt that cheap.
flattening and especially framing are the most expensive, and expansion is the cheapest especially since all the other algorithms depend on it (though if you do expand manually before it'll take a fast path out)
my argument here is that, if you know the structure you're serializing to (i.e. if you're a contemporary AP implementation that isn't doing anything too fancy), you can directly serialize in compacted form and skip constructing a tree of JSON objects in your library and running the compaction algorithm over it. depending on how clever you(r libraries) get you may be able to directly write the JSON string directly, even.
from some brief profiling i've done this does show up as a hot code path in iceshrimp.net, one of my goals with Eventually replacing dotNetRdf with my own impl mentioned above is to, given i'm gonna have to mess with serialization anyhow, remove the JSON-LD bits there and serialize directly to compacted form which should help with large boosts and other bursts -
@hongminhee from the point of view of someone who is "maintaining" a JSON-LD processing fedi software and has implemented their own JSON-LD processing library (which is, to my knowledge, the fastest in it's programming language), JSON-LD is pure overhead. there is nothing it allows for that can't be done with
1. making fields which take multiple values explicit
2. always using namespaces and letting HTTP compression take care of minimizing the transfer
without JSON-LD, fedi software could use zero-ish-copy deserialization for a majority of their objects (when strings aren't escaped) through tools like serde_json and Cow<str>, or System.Text.Json.JsonDocument. JSON-LD processing effectively mandates a JSON node DOM (in the algorithms standardized, you may be able to get rid of it with Clever Programming)
additionally, due to JSON-LD 1.1 features like @type:@json, you can not even fetch contexts in parallel, meaning all JSON-LD code has to be async (in the languages which has the concept), potentially losing out on significant optimizations that can't be done in coroutines due to various reasons (e.g. C# async methods can't have ref structs, Rust async functions usually require thread safety due to tokio's prevalence, even if they're ran in a single-threaded runtime)
this is after context processing introducing network dependency to the deserialization of data, wasting time and data on non-server cases (e.g. activitypub C2S). sure you can cache individual contexts, but then the context can change underneath you, desynchronizing your cached context and, in the worst case, opening you up to security vulnerabilities
json-ld is not my favorite part of this protocol@kopper @hongminhee As the person probably most responsible for making sure json-ld stayed in the spec (two reasons: because it was the only extensibility answer we had, and because we were trying hard to retain interoperability with the linked data people, which ultimately did not matter), I agree with you. I do ultimately regret not having a simpler solution than json-ld, especially because it greatly hurt our ability to sign messages, which has considerable effect on the ecosystem.
Mea culpa
![😕](https://postcall.pub/assets/plugins/nodebb-plugin-emoji/emoji/android/1f615.png?v=2716ecfffdc ":\")
I do think it's fixable. I'd be interested in joining a conversation about how to fix it.
-
@kopper @hongminhee As the person probably most responsible for making sure json-ld stayed in the spec (two reasons: because it was the only extensibility answer we had, and because we were trying hard to retain interoperability with the linked data people, which ultimately did not matter), I agree with you. I do ultimately regret not having a simpler solution than json-ld, especially because it greatly hurt our ability to sign messages, which has considerable effect on the ecosystem.
Mea culpa
I do think it's fixable. I'd be interested in joining a conversation about how to fix it.
I don't remember it that way.
We started the WG off with AS2 being based on JSON-LD, and I don't think we ever considered removing it.
I don't think it was a decision you made on your own. I'm not sure how you would, since you edited AP and not AS2 Core or Vocabulary.
-
I don't remember it that way.
We started the WG off with AS2 being based on JSON-LD, and I don't think we ever considered removing it.
I don't think it was a decision you made on your own. I'm not sure how you would, since you edited AP and not AS2 Core or Vocabulary.
I would be strongly opposed to any effort to remove JSON-LD from AS2. We use it for a lot of extensions. Every AP server uses the Security vocabulary for public keys.
-
I would be strongly opposed to any effort to remove JSON-LD from AS2. We use it for a lot of extensions. Every AP server uses the Security vocabulary for public keys.
@cwebber @kopper @hongminhee It would be a huge backwards-incompatible change for almost zero benefit. People would still make mistakes in their ActivityPub implementations (sorry, Minhee, but that's life on an open network). We'd need to adopt another mechanism for defining extensions, and guess what? People are going to make mistakes with that, too.
-
@cwebber @kopper @hongminhee It would be a huge backwards-incompatible change for almost zero benefit. People would still make mistakes in their ActivityPub implementations (sorry, Minhee, but that's life on an open network). We'd need to adopt another mechanism for defining extensions, and guess what? People are going to make mistakes with that, too.
@cwebber @kopper @hongminhee The biggest downside to JSON-LD, it seems, is that it lets most developers treat AS2 as if it's plain old JSON. That was by design. People sometimes mess it up, but most JSON-LD parsers are pretty tolerant.
-
@cwebber @kopper @hongminhee The biggest downside to JSON-LD, it seems, is that it lets most developers treat AS2 as if it's plain old JSON. That was by design. People sometimes mess it up, but most JSON-LD parsers are pretty tolerant.
@evan @cwebber @kopper @hongminhee Couldn’t we agree to standardize on expanded json-ld? We would not need any json-ld processor, we would not need to fetch or cache any context. There would be no way to shadow properties.
-
@evan @cwebber @kopper @hongminhee Couldn’t we agree to standardize on expanded json-ld? We would not need any json-ld processor, we would not need to fetch or cache any context. There would be no way to shadow properties.
@gugurumbe @cwebber @kopper @hongminhee AS2 requires compacted JSON-LD.
-
@gugurumbe @cwebber @kopper @hongminhee AS2 requires compacted JSON-LD.
@evan @gugurumbe @cwebber @kopper @hongminhee only for terms defined in AS2, though?
if the activitystreams context is missing in an application/activity+json document, then you MUST assume/inject it. this means you can't redefine "actor" to mean "actor in a movie".
otherwise, you don't have to augment the context with anything else. "https://w3id.org/security#publicKey" is a valid property name. the proposal is to not augment the normative context where possible. no parsing context if there is no context
-
@evan @gugurumbe @cwebber @kopper @hongminhee only for terms defined in AS2, though?
if the activitystreams context is missing in an application/activity+json document, then you MUST assume/inject it. this means you can't redefine "actor" to mean "actor in a movie".
otherwise, you don't have to augment the context with anything else. "https://w3id.org/security#publicKey" is a valid property name. the proposal is to not augment the normative context where possible. no parsing context if there is no context
@trwnh i was replying to a post that wanted all expanded terms.
-
@trwnh i was replying to a post that wanted all expanded terms.
@evan @trwnh @cwebber @kopper @hongminhee I think it would be great to have everything expanded besides the required as2 context.
The results of the compaction algorithm would change if new things migrate into schema.org, so technically a document could become invalid or break without being modified, but this would be a lot better otherwise I guess. -
@evan @trwnh @cwebber @kopper @hongminhee I think it would be great to have everything expanded besides the required as2 context.
The results of the compaction algorithm would change if new things migrate into schema.org, so technically a document could become invalid or break without being modified, but this would be a lot better otherwise I guess.@gugurumbe @evan @cwebber @kopper @hongminhee yup, using full IRIs also has the advantage that ld-unaware processors only need to recognize 1 form instead of infinitely many.
the thing is, we have semantics imported from the content type (activity+json) which can also change. which is why i think versioning the context document is also important -- it freezes the semantics at the time of publishing, like pinning your dependencies.
without that, we might well have a simpler profile...
-
@gugurumbe @evan @cwebber @kopper @hongminhee yup, using full IRIs also has the advantage that ld-unaware processors only need to recognize 1 form instead of infinitely many.
the thing is, we have semantics imported from the content type (activity+json) which can also change. which is why i think versioning the context document is also important -- it freezes the semantics at the time of publishing, like pinning your dependencies.
without that, we might well have a simpler profile...
@trwnh as a paranoid person, I sometimes wonder what would happen if schema.org received a court order to partially censor its schema in certain regions of the world. Or inject a backdoor key. If it prevents people from sending memes across the geofence, it’s bad.
-
R AodeRelay shared this topic
