I have deeply mixed feelings about #ActivityPub's adoption of JSON-LD, as someone who's spent way too long dealing with it while building #Fedify.
-
Do you know about the backgrounds of the immers project ?
, "no-one's actually using AP to play chess"
the reason that we have noa AP chess service _anymore_ is #uspol …This all feels very unfair somehow cause I know the backgrounds but anyway …
While we 2 days ago had a long thread about our use of Chess Games I will link the video from the thread https://digitalcourage.social/@sl007/116023149133783002immers with its federated locations and positional audio etc was supernice for playing chess !
Our use is fairly similar and straightforward like we did the chess Social CG meeting in 2018 and the rc3 (usually 18.000 people physically but here it was virtually cause pandemics) https://socialhub.activitypub.rocks/t/rc3-chaos-communication-congress/1202Maybe it would really be fair if people are new to look into the 20 years Social CG history where some volunteers really gave much work
🧵 1/2We implemented this standard and you can create / describe your rooms [Place, `redaktor:fictional`] and the chessboard is just a geohash as described in the geosocial CG so the use is the same, just `redaktor:fictional` too,
You load the Collection of Chessfigures (pawn1 ...) can name them, they `Travel` over the chessboard ant the `Arrive` describes the `result`.
As always you can get very detailed with wikidata properties and entities but bare AS Vocabulary is enough.
In the end you have a Collection for the Travels which is your played game which you can replay or do whatever with.But you can still install immers - it is worth a try https://github.com/immers-space
The reason for its end are the same as for the gup.pe groups and I hope people konw about it …
-
We implemented this standard and you can create / describe your rooms [Place, `redaktor:fictional`] and the chessboard is just a geohash as described in the geosocial CG so the use is the same, just `redaktor:fictional` too,
You load the Collection of Chessfigures (pawn1 ...) can name them, they `Travel` over the chessboard ant the `Arrive` describes the `result`.
As always you can get very detailed with wikidata properties and entities but bare AS Vocabulary is enough.
In the end you have a Collection for the Travels which is your played game which you can replay or do whatever with.But you can still install immers - it is worth a try https://github.com/immers-space
The reason for its end are the same as for the gup.pe groups and I hope people konw about it …
@sl007 @julian I admit I didn't pay attention to immers at the time - I don't play games, not even chess. I was just using chess as an example, didn't mean to trigger anyone's trauma!
Still, it kinda proves my point. You have to use standard AS vocabulary because Mastodon, and if you squint then sure,
TravelandArrive, why not? But given some of the conversations I've seen on this forum, I shudder to think how that would go down if you tried to get approval for that usage from "the community" first. -
@hongminhee @jalefkowit huh. I’ve been pondering using it for some projects of mine, so this is good to know.
Is it a fundamental problem with JSON-LD, such that it should just be avoided, or a problem with how ActivityPub uses it?
And is there something else you’d recommend that fulfills the same goals?
-
@hongminhee @jalefkowit huh. I’ve been pondering using it for some projects of mine, so this is good to know.
Is it a fundamental problem with JSON-LD, such that it should just be avoided, or a problem with how ActivityPub uses it?
And is there something else you’d recommend that fulfills the same goals?
@lkanies@hachyderm.io @jalefkowit@vmst.io To be honest, I'm not too sure myself. I just know that JSON-LD was originally planned as a foundation for the Semantic Web. I can only guess that if ontology is useful in a certain area, then JSON-LD would probably be useful there too.
-
@hongminhee do you use the activitystrea.ms module from npm? It takes a lot of the pain out.
-
@hongminhee do you use the activitystrea.ms module from npm? It takes a lot of the pain out.
@evan@cosocial.ca I don't remember exactly, but I think I came across it while doing research before developing Fedify. I probably didn't use it because the TypeScript type definitions were missing. In the end, I ended up making something similar in Fedify anyway.
-
@pintoch read this thread?
-
@hongminhee from the point of view of someone who is "maintaining" a JSON-LD processing fedi software and has implemented their own JSON-LD processing library (which is, to my knowledge, the fastest in it's programming language), JSON-LD is pure overhead. there is nothing it allows for that can't be done with
1. making fields which take multiple values explicit
2. always using namespaces and letting HTTP compression take care of minimizing the transfer
without JSON-LD, fedi software could use zero-ish-copy deserialization for a majority of their objects (when strings aren't escaped) through tools like serde_json and Cow<str>, or System.Text.Json.JsonDocument. JSON-LD processing effectively mandates a JSON node DOM (in the algorithms standardized, you may be able to get rid of it with Clever Programming)
additionally, due to JSON-LD 1.1 features like @type:@json, you can not even fetch contexts in parallel, meaning all JSON-LD code has to be async (in the languages which has the concept), potentially losing out on significant optimizations that can't be done in coroutines due to various reasons (e.g. C# async methods can't have ref structs, Rust async functions usually require thread safety due to tokio's prevalence, even if they're ran in a single-threaded runtime)
this is after context processing introducing network dependency to the deserialization of data, wasting time and data on non-server cases (e.g. activitypub C2S). sure you can cache individual contexts, but then the context can change underneath you, desynchronizing your cached context and, in the worst case, opening you up to security vulnerabilities
json-ld is not my favorite part of this protocol -
@hongminhee from the point of view of someone who is "maintaining" a JSON-LD processing fedi software and has implemented their own JSON-LD processing library (which is, to my knowledge, the fastest in it's programming language), JSON-LD is pure overhead. there is nothing it allows for that can't be done with
1. making fields which take multiple values explicit
2. always using namespaces and letting HTTP compression take care of minimizing the transfer
without JSON-LD, fedi software could use zero-ish-copy deserialization for a majority of their objects (when strings aren't escaped) through tools like serde_json and Cow<str>, or System.Text.Json.JsonDocument. JSON-LD processing effectively mandates a JSON node DOM (in the algorithms standardized, you may be able to get rid of it with Clever Programming)
additionally, due to JSON-LD 1.1 features like @type:@json, you can not even fetch contexts in parallel, meaning all JSON-LD code has to be async (in the languages which has the concept), potentially losing out on significant optimizations that can't be done in coroutines due to various reasons (e.g. C# async methods can't have ref structs, Rust async functions usually require thread safety due to tokio's prevalence, even if they're ran in a single-threaded runtime)
this is after context processing introducing network dependency to the deserialization of data, wasting time and data on non-server cases (e.g. activitypub C2S). sure you can cache individual contexts, but then the context can change underneath you, desynchronizing your cached context and, in the worst case, opening you up to security vulnerabilities
json-ld is not my favorite part of this protocol@hongminhee take this part with a grain of salt because my benchmarks for it are with dotNetRdf which is the slowest C# implementation i know of (hence my replacement library), but JSON-LD is slower than RSA validation, which is one of the pain points around authorized fetch scalability
wetdry.world/@kopper/114678924693500011 -
@hongminhee take this part with a grain of salt because my benchmarks for it are with dotNetRdf which is the slowest C# implementation i know of (hence my replacement library), but JSON-LD is slower than RSA validation, which is one of the pain points around authorized fetch scalability
wetdry.world/@kopper/114678924693500011@hongminhee if i can give one piece of advice to devs who want to process JSON-LD: dont bother compacting. you already know the schema you output (or you're just passing through what the user gives and it doesn't matter to you), serialize directly to the compacted representation, and only run expansion on incoming data
expansion is the cheapest JSON-LD operation (since all other operations depend on it and run it internally anyhow), and this will get you all the compatibility benefits of JSON-LD with little downsides (beyond more annoying deserialization code, as you have to map the expanded representation to your internal structure which will likely be modeled after the compacted one) -
@hongminhee if i can give one piece of advice to devs who want to process JSON-LD: dont bother compacting. you already know the schema you output (or you're just passing through what the user gives and it doesn't matter to you), serialize directly to the compacted representation, and only run expansion on incoming data
expansion is the cheapest JSON-LD operation (since all other operations depend on it and run it internally anyhow), and this will get you all the compatibility benefits of JSON-LD with little downsides (beyond more annoying deserialization code, as you have to map the expanded representation to your internal structure which will likely be modeled after the compacted one)@kopper@not-brain.d.on-t.work @hongminhee@hollo.social expansion is actually really annoying because the resulting JSON has annoyingly similar keys to lookup in a hashmap, wasting cache lines, and CPU time
-
@kopper@not-brain.d.on-t.work @hongminhee@hollo.social expansion is actually really annoying because the resulting JSON has annoyingly similar keys to lookup in a hashmap, wasting cache lines, and CPU time
@natty @hongminhee i would imagine a Good hash algorithm wouldn't care about the similarity of the keys, no? -
@hongminhee take this part with a grain of salt because my benchmarks for it are with dotNetRdf which is the slowest C# implementation i know of (hence my replacement library), but JSON-LD is slower than RSA validation, which is one of the pain points around authorized fetch scalability
wetdry.world/@kopper/114678924693500011@hongminhee i put this in a quote but people reading the thread may also be interested: json-ld compaction does not really save that much bandwidth over having all the prefixes explicitly defined if you're gzipping (and you are gzipping, right? this is json. make sure your nginx gzip_types includes ld+json and activity+json)
RE: not-brain.d.on-t.work/notes/aihftmbjpxdyb9k7 -
@hongminhee from the point of view of someone who is "maintaining" a JSON-LD processing fedi software and has implemented their own JSON-LD processing library (which is, to my knowledge, the fastest in it's programming language), JSON-LD is pure overhead. there is nothing it allows for that can't be done with
1. making fields which take multiple values explicit
2. always using namespaces and letting HTTP compression take care of minimizing the transfer
without JSON-LD, fedi software could use zero-ish-copy deserialization for a majority of their objects (when strings aren't escaped) through tools like serde_json and Cow<str>, or System.Text.Json.JsonDocument. JSON-LD processing effectively mandates a JSON node DOM (in the algorithms standardized, you may be able to get rid of it with Clever Programming)
additionally, due to JSON-LD 1.1 features like @type:@json, you can not even fetch contexts in parallel, meaning all JSON-LD code has to be async (in the languages which has the concept), potentially losing out on significant optimizations that can't be done in coroutines due to various reasons (e.g. C# async methods can't have ref structs, Rust async functions usually require thread safety due to tokio's prevalence, even if they're ran in a single-threaded runtime)
this is after context processing introducing network dependency to the deserialization of data, wasting time and data on non-server cases (e.g. activitypub C2S). sure you can cache individual contexts, but then the context can change underneath you, desynchronizing your cached context and, in the worst case, opening you up to security vulnerabilities
json-ld is not my favorite part of this protocolhm, we really need to differentiate between users responsibility and dev responsibility.
Not sure if Hong saw the draft about the AP kv thing, it supports either JSON-LD fields _or_ as:attachment / as:context …
wtf do I want to say.user story:
We are working on 2 major and 3 projects fulltime which is
- federation of wikibase / wikidata
- federation of Public Broadcasters https://www.publicmediaalliance.org/public-broadcasters-create-public-spaces-incubator/
and these https://codeberg.org/Menschys/fedi-codebaseLet's say we want to federate a Country, then all the knowledge is sent in `attachment` with the fully qualified qikidata url in `context` [as:context - not @context ! - this is so confusing :)]
For example the according entries from the PressFreedomIndex `collection` (co-founder of freelens hereBut anyway, the idea about having
"wd": "https://www.wikidata.org/wiki/Special:EntityData/",
"wdt": "https://www.wikidata.org/prop/direct/" in the `@context` was that any user can consume and federate wikibase
incl.
🧵 1/2 -
hm, we really need to differentiate between users responsibility and dev responsibility.
Not sure if Hong saw the draft about the AP kv thing, it supports either JSON-LD fields _or_ as:attachment / as:context …
wtf do I want to say.user story:
We are working on 2 major and 3 projects fulltime which is
- federation of wikibase / wikidata
- federation of Public Broadcasters https://www.publicmediaalliance.org/public-broadcasters-create-public-spaces-incubator/
and these https://codeberg.org/Menschys/fedi-codebaseLet's say we want to federate a Country, then all the knowledge is sent in `attachment` with the fully qualified qikidata url in `context` [as:context - not @context ! - this is so confusing :)]
For example the according entries from the PressFreedomIndex `collection` (co-founder of freelens hereBut anyway, the idea about having
"wd": "https://www.wikidata.org/wiki/Special:EntityData/",
"wdt": "https://www.wikidata.org/prop/direct/" in the `@context` was that any user can consume and federate wikibase
incl.
🧵 1/2incl.
- the properties in all the languages of the world
- the knowledge of the world in all the languages
- the wikidata relations and qualified statements including the nameMap etc. and all the urls to all wikiprojects incl. their languages and knowledgeHow else could I say to other softwares if they want all users qualified data, use wikidata vocabulary?
wikipedia, wikidata, EBU, Public Broadcasters, taxi data is _all_ JSON-LD … -
incl.
- the properties in all the languages of the world
- the knowledge of the world in all the languages
- the wikidata relations and qualified statements including the nameMap etc. and all the urls to all wikiprojects incl. their languages and knowledgeHow else could I say to other softwares if they want all users qualified data, use wikidata vocabulary?
wikipedia, wikidata, EBU, Public Broadcasters, taxi data is _all_ JSON-LD …@sl007 @hongminhee @julian i feel like you're falling into a trap i've seen a lot around AP spaces: just because the data can be contorted to represent something does not mean software will interpret it as such.
any software who wants to support wikidata statements and relations will have to go out of their way to implement that manually with or without json-ld in the mix, and interoperability between those software will have to specify how that works. and in your specification you can indeed make it so Simply Linking to the wikidata json-ld (which i don't believe it provides out of the box, it does for xml and n-triples, if we're talking about rdf. if not, their bespoke json format is just as authoritative) can work (but i'd say using the Qxxx and Pxx IDs and letting the software figure out how to access it would be better!)
if you have the dream of making an as:Note and having it's as:attributedTo be the wikidata entity for alan turing... sorry, nobody other than maybe your own software will even attempt interpreting that -
R AodeRelay shared this topic
-
@sl007 @hongminhee @julian i feel like you're falling into a trap i've seen a lot around AP spaces: just because the data can be contorted to represent something does not mean software will interpret it as such.
any software who wants to support wikidata statements and relations will have to go out of their way to implement that manually with or without json-ld in the mix, and interoperability between those software will have to specify how that works. and in your specification you can indeed make it so Simply Linking to the wikidata json-ld (which i don't believe it provides out of the box, it does for xml and n-triples, if we're talking about rdf. if not, their bespoke json format is just as authoritative) can work (but i'd say using the Qxxx and Pxx IDs and letting the software figure out how to access it would be better!)
if you have the dream of making an as:Note and having it's as:attributedTo be the wikidata entity for alan turing... sorry, nobody other than maybe your own software will even attempt interpreting that@hongminhee @sl007 @julian attempting to support this kind of "data contortion" (i made this up and prolly isnt the right way to describe this) would rapidly balloon the scope of every fedi software ever. i don't believe anyone would want to develop for such ecosystem
a similar example i saw was someone attempting to explain how you can partially inline an as:object you as:Like'd in order to specify you only liked that past version of it and if it changed your like shouldn't count. without describing this exact scenario i don't believe any software, json-ld capable or not, would interpret that Like as such. same thing with the long-form text FEP which attempts to support non-activitypub authors -
@hongminhee @sl007 @julian attempting to support this kind of "data contortion" (i made this up and prolly isnt the right way to describe this) would rapidly balloon the scope of every fedi software ever. i don't believe anyone would want to develop for such ecosystem
a similar example i saw was someone attempting to explain how you can partially inline an as:object you as:Like'd in order to specify you only liked that past version of it and if it changed your like shouldn't count. without describing this exact scenario i don't believe any software, json-ld capable or not, would interpret that Like as such. same thing with the long-form text FEP which attempts to support non-activitypub authorsit is just damned simple, your as: Client can do so much by asking wikidata, OSM, federated geocoding and not our system. When you use a property for the first time, the client can cache its names in the languages of the user etc.
-
it is just damned simple, your as: Client can do so much by asking wikidata, OSM, federated geocoding and not our system. When you use a property for the first time, the client can cache its names in the languages of the user etc.
@sl007 @hongminhee @julian i genuinely can't see where json-ld is relevant here. if your client wants to support wikidata and OSM then it can do that with or without json-ld being involved. you are going to have to document how this integration works anyhow if you want anyone else to do so -
@hongminhee How hard would it be for a future version of ActivityPub to simply back out JSON-LD support? Would there be a downside to this?
