I have deeply mixed feelings about #ActivityPub's adoption of JSON-LD, as someone who's spent way too long dealing with it while building #Fedify.
-
> while linked data cultists harass developers about nonresolvable URLs
@silverpill I don't consider myself a cultist but I still think that putting invalid URLs in any payload where they are supposed to be meaningful is disrespectful towards anyone that consumes your API. Please don't do that.
-
@hongminhee@hollo.social I'll give you my take on this... which is that my understanding of JSON-LD is that with JSON-LD you can have two disparate apps using the same property, like
thread, and avoid namespace collision because one is actuallyhttps://example.org/ns/threadand the other's reallyhttps://foobar.com/ns/thread.Great.
I posit that this is a premature optimization, and one that fails because of inadequate adoption. There are likely documented cases of implementations using the same property, and those concern the actual ActivityStreams vocabulary, and the solution to that is to communicate and work together so that you don't step on each others' toes.
I personally feel that it is a technical solution to a problem that can be completely handled by simply talking to one another... but we're coders, we're famously anti-social yes? mmmmm...
-
@hongminhee@hollo.social I'll give you my take on this... which is that my understanding of JSON-LD is that with JSON-LD you can have two disparate apps using the same property, like
thread, and avoid namespace collision because one is actuallyhttps://example.org/ns/threadand the other's reallyhttps://foobar.com/ns/thread.Great.
I posit that this is a premature optimization, and one that fails because of inadequate adoption. There are likely documented cases of implementations using the same property, and those concern the actual ActivityStreams vocabulary, and the solution to that is to communicate and work together so that you don't step on each others' toes.
I personally feel that it is a technical solution to a problem that can be completely handled by simply talking to one another... but we're coders, we're famously anti-social yes? mmmmm...
@hongminhee @julian I'm a true believer in RDF from back in the day, so I'm hardly neutral. But...
There are essentially no interesting ActivityPub extensions right now. Even Evan's chess example, no-one's actually using AP to play chess. It's just ActivityStreams + a few cute tricks now and then. Even if there were extensions, existing AP servers chop off and throw away data they don't understand. so none of these extensions could work.
I feel like most of the "WTF am I learning JSON-LD for" criticisms are coming from this status quo. That includes "if someone wants to add a gallery thing or whatever, can't they make a FEP?" The way things work now, your extension either a) works only in your software or b) has to be painfully negotiated with the whole community. We're all gonna have a big fight about it on this forum anyway. Let's not pretend JSON-LD helps us.
But if we add two things to the mix, the situation looks different. Those are 1. server software that "keeps all the bits", and 2. a whitelabel extensible app. That would make it very easy to spin up crazy new experiences for a sizeable existing userbase. Developers should not be forced to endure a FEP process, and they should not have to attract a userbase from nothing. They should be able to just build, without even worrying if they're stepping on toes. And of course, Fedify and libraries in other languages are a load-bearing part of that world, including enforcement of the JSON-LD rules.
That world does not exist at all today, but JSON-LD does, so it's pretty valid to describe this design as premature optimisation. I dunno though, we don't seem that far away.
-
@hongminhee @julian I'm a true believer in RDF from back in the day, so I'm hardly neutral. But...
There are essentially no interesting ActivityPub extensions right now. Even Evan's chess example, no-one's actually using AP to play chess. It's just ActivityStreams + a few cute tricks now and then. Even if there were extensions, existing AP servers chop off and throw away data they don't understand. so none of these extensions could work.
I feel like most of the "WTF am I learning JSON-LD for" criticisms are coming from this status quo. That includes "if someone wants to add a gallery thing or whatever, can't they make a FEP?" The way things work now, your extension either a) works only in your software or b) has to be painfully negotiated with the whole community. We're all gonna have a big fight about it on this forum anyway. Let's not pretend JSON-LD helps us.
But if we add two things to the mix, the situation looks different. Those are 1. server software that "keeps all the bits", and 2. a whitelabel extensible app. That would make it very easy to spin up crazy new experiences for a sizeable existing userbase. Developers should not be forced to endure a FEP process, and they should not have to attract a userbase from nothing. They should be able to just build, without even worrying if they're stepping on toes. And of course, Fedify and libraries in other languages are a load-bearing part of that world, including enforcement of the JSON-LD rules.
That world does not exist at all today, but JSON-LD does, so it's pretty valid to describe this design as premature optimisation. I dunno though, we don't seem that far away.
@mat@friendica.exon.name that's a really interesting point of view, and has some parallels to how app development on the ATProto side is easier in many ways.
I do think that this is something C2S (aka the ActivityPub API) can enable.
I am critical of JSON-LD but I do certainly recognize I could be very wrong
-
@mat@friendica.exon.name that's a really interesting point of view, and has some parallels to how app development on the ATProto side is easier in many ways.
I do think that this is something C2S (aka the ActivityPub API) can enable.
I am critical of JSON-LD but I do certainly recognize I could be very wrong
Do you know about the backgrounds of the immers project ?
, "no-one's actually using AP to play chess"
the reason that we have noa AP chess service _anymore_ is #uspol …This all feels very unfair somehow cause I know the backgrounds but anyway …
While we 2 days ago had a long thread about our use of Chess Games I will link the video from the thread https://digitalcourage.social/@sl007/116023149133783002immers with its federated locations and positional audio etc was supernice for playing chess !
Our use is fairly similar and straightforward like we did the chess Social CG meeting in 2018 and the rc3 (usually 18.000 people physically but here it was virtually cause pandemics) https://socialhub.activitypub.rocks/t/rc3-chaos-communication-congress/1202Maybe it would really be fair if people are new to look into the 20 years Social CG history where some volunteers really gave much work
🧵 1/2 -
Do you know about the backgrounds of the immers project ?
, "no-one's actually using AP to play chess"
the reason that we have noa AP chess service _anymore_ is #uspol …This all feels very unfair somehow cause I know the backgrounds but anyway …
While we 2 days ago had a long thread about our use of Chess Games I will link the video from the thread https://digitalcourage.social/@sl007/116023149133783002immers with its federated locations and positional audio etc was supernice for playing chess !
Our use is fairly similar and straightforward like we did the chess Social CG meeting in 2018 and the rc3 (usually 18.000 people physically but here it was virtually cause pandemics) https://socialhub.activitypub.rocks/t/rc3-chaos-communication-congress/1202Maybe it would really be fair if people are new to look into the 20 years Social CG history where some volunteers really gave much work
🧵 1/2We implemented this standard and you can create / describe your rooms [Place, `redaktor:fictional`] and the chessboard is just a geohash as described in the geosocial CG so the use is the same, just `redaktor:fictional` too,
You load the Collection of Chessfigures (pawn1 ...) can name them, they `Travel` over the chessboard ant the `Arrive` describes the `result`.
As always you can get very detailed with wikidata properties and entities but bare AS Vocabulary is enough.
In the end you have a Collection for the Travels which is your played game which you can replay or do whatever with.But you can still install immers - it is worth a try https://github.com/immers-space
The reason for its end are the same as for the gup.pe groups and I hope people konw about it …
-
We implemented this standard and you can create / describe your rooms [Place, `redaktor:fictional`] and the chessboard is just a geohash as described in the geosocial CG so the use is the same, just `redaktor:fictional` too,
You load the Collection of Chessfigures (pawn1 ...) can name them, they `Travel` over the chessboard ant the `Arrive` describes the `result`.
As always you can get very detailed with wikidata properties and entities but bare AS Vocabulary is enough.
In the end you have a Collection for the Travels which is your played game which you can replay or do whatever with.But you can still install immers - it is worth a try https://github.com/immers-space
The reason for its end are the same as for the gup.pe groups and I hope people konw about it …
@sl007 @julian I admit I didn't pay attention to immers at the time - I don't play games, not even chess. I was just using chess as an example, didn't mean to trigger anyone's trauma!
Still, it kinda proves my point. You have to use standard AS vocabulary because Mastodon, and if you squint then sure,
TravelandArrive, why not? But given some of the conversations I've seen on this forum, I shudder to think how that would go down if you tried to get approval for that usage from "the community" first. -
@hongminhee @jalefkowit huh. I’ve been pondering using it for some projects of mine, so this is good to know.
Is it a fundamental problem with JSON-LD, such that it should just be avoided, or a problem with how ActivityPub uses it?
And is there something else you’d recommend that fulfills the same goals?
-
@hongminhee @jalefkowit huh. I’ve been pondering using it for some projects of mine, so this is good to know.
Is it a fundamental problem with JSON-LD, such that it should just be avoided, or a problem with how ActivityPub uses it?
And is there something else you’d recommend that fulfills the same goals?
@lkanies@hachyderm.io @jalefkowit@vmst.io To be honest, I'm not too sure myself. I just know that JSON-LD was originally planned as a foundation for the Semantic Web. I can only guess that if ontology is useful in a certain area, then JSON-LD would probably be useful there too.
-
@hongminhee do you use the activitystrea.ms module from npm? It takes a lot of the pain out.
-
@hongminhee do you use the activitystrea.ms module from npm? It takes a lot of the pain out.
@evan@cosocial.ca I don't remember exactly, but I think I came across it while doing research before developing Fedify. I probably didn't use it because the TypeScript type definitions were missing. In the end, I ended up making something similar in Fedify anyway.
-
@pintoch read this thread?
-
@hongminhee from the point of view of someone who is "maintaining" a JSON-LD processing fedi software and has implemented their own JSON-LD processing library (which is, to my knowledge, the fastest in it's programming language), JSON-LD is pure overhead. there is nothing it allows for that can't be done with
1. making fields which take multiple values explicit
2. always using namespaces and letting HTTP compression take care of minimizing the transfer
without JSON-LD, fedi software could use zero-ish-copy deserialization for a majority of their objects (when strings aren't escaped) through tools like serde_json and Cow<str>, or System.Text.Json.JsonDocument. JSON-LD processing effectively mandates a JSON node DOM (in the algorithms standardized, you may be able to get rid of it with Clever Programming)
additionally, due to JSON-LD 1.1 features like @type:@json, you can not even fetch contexts in parallel, meaning all JSON-LD code has to be async (in the languages which has the concept), potentially losing out on significant optimizations that can't be done in coroutines due to various reasons (e.g. C# async methods can't have ref structs, Rust async functions usually require thread safety due to tokio's prevalence, even if they're ran in a single-threaded runtime)
this is after context processing introducing network dependency to the deserialization of data, wasting time and data on non-server cases (e.g. activitypub C2S). sure you can cache individual contexts, but then the context can change underneath you, desynchronizing your cached context and, in the worst case, opening you up to security vulnerabilities
json-ld is not my favorite part of this protocol -
@hongminhee from the point of view of someone who is "maintaining" a JSON-LD processing fedi software and has implemented their own JSON-LD processing library (which is, to my knowledge, the fastest in it's programming language), JSON-LD is pure overhead. there is nothing it allows for that can't be done with
1. making fields which take multiple values explicit
2. always using namespaces and letting HTTP compression take care of minimizing the transfer
without JSON-LD, fedi software could use zero-ish-copy deserialization for a majority of their objects (when strings aren't escaped) through tools like serde_json and Cow<str>, or System.Text.Json.JsonDocument. JSON-LD processing effectively mandates a JSON node DOM (in the algorithms standardized, you may be able to get rid of it with Clever Programming)
additionally, due to JSON-LD 1.1 features like @type:@json, you can not even fetch contexts in parallel, meaning all JSON-LD code has to be async (in the languages which has the concept), potentially losing out on significant optimizations that can't be done in coroutines due to various reasons (e.g. C# async methods can't have ref structs, Rust async functions usually require thread safety due to tokio's prevalence, even if they're ran in a single-threaded runtime)
this is after context processing introducing network dependency to the deserialization of data, wasting time and data on non-server cases (e.g. activitypub C2S). sure you can cache individual contexts, but then the context can change underneath you, desynchronizing your cached context and, in the worst case, opening you up to security vulnerabilities
json-ld is not my favorite part of this protocol@hongminhee take this part with a grain of salt because my benchmarks for it are with dotNetRdf which is the slowest C# implementation i know of (hence my replacement library), but JSON-LD is slower than RSA validation, which is one of the pain points around authorized fetch scalability
wetdry.world/@kopper/114678924693500011 -
@hongminhee take this part with a grain of salt because my benchmarks for it are with dotNetRdf which is the slowest C# implementation i know of (hence my replacement library), but JSON-LD is slower than RSA validation, which is one of the pain points around authorized fetch scalability
wetdry.world/@kopper/114678924693500011@hongminhee if i can give one piece of advice to devs who want to process JSON-LD: dont bother compacting. you already know the schema you output (or you're just passing through what the user gives and it doesn't matter to you), serialize directly to the compacted representation, and only run expansion on incoming data
expansion is the cheapest JSON-LD operation (since all other operations depend on it and run it internally anyhow), and this will get you all the compatibility benefits of JSON-LD with little downsides (beyond more annoying deserialization code, as you have to map the expanded representation to your internal structure which will likely be modeled after the compacted one) -
@hongminhee if i can give one piece of advice to devs who want to process JSON-LD: dont bother compacting. you already know the schema you output (or you're just passing through what the user gives and it doesn't matter to you), serialize directly to the compacted representation, and only run expansion on incoming data
expansion is the cheapest JSON-LD operation (since all other operations depend on it and run it internally anyhow), and this will get you all the compatibility benefits of JSON-LD with little downsides (beyond more annoying deserialization code, as you have to map the expanded representation to your internal structure which will likely be modeled after the compacted one)@kopper@not-brain.d.on-t.work @hongminhee@hollo.social expansion is actually really annoying because the resulting JSON has annoyingly similar keys to lookup in a hashmap, wasting cache lines, and CPU time
-
@kopper@not-brain.d.on-t.work @hongminhee@hollo.social expansion is actually really annoying because the resulting JSON has annoyingly similar keys to lookup in a hashmap, wasting cache lines, and CPU time
@natty @hongminhee i would imagine a Good hash algorithm wouldn't care about the similarity of the keys, no? -
@hongminhee take this part with a grain of salt because my benchmarks for it are with dotNetRdf which is the slowest C# implementation i know of (hence my replacement library), but JSON-LD is slower than RSA validation, which is one of the pain points around authorized fetch scalability
wetdry.world/@kopper/114678924693500011@hongminhee i put this in a quote but people reading the thread may also be interested: json-ld compaction does not really save that much bandwidth over having all the prefixes explicitly defined if you're gzipping (and you are gzipping, right? this is json. make sure your nginx gzip_types includes ld+json and activity+json)
RE: not-brain.d.on-t.work/notes/aihftmbjpxdyb9k7 -
@hongminhee from the point of view of someone who is "maintaining" a JSON-LD processing fedi software and has implemented their own JSON-LD processing library (which is, to my knowledge, the fastest in it's programming language), JSON-LD is pure overhead. there is nothing it allows for that can't be done with
1. making fields which take multiple values explicit
2. always using namespaces and letting HTTP compression take care of minimizing the transfer
without JSON-LD, fedi software could use zero-ish-copy deserialization for a majority of their objects (when strings aren't escaped) through tools like serde_json and Cow<str>, or System.Text.Json.JsonDocument. JSON-LD processing effectively mandates a JSON node DOM (in the algorithms standardized, you may be able to get rid of it with Clever Programming)
additionally, due to JSON-LD 1.1 features like @type:@json, you can not even fetch contexts in parallel, meaning all JSON-LD code has to be async (in the languages which has the concept), potentially losing out on significant optimizations that can't be done in coroutines due to various reasons (e.g. C# async methods can't have ref structs, Rust async functions usually require thread safety due to tokio's prevalence, even if they're ran in a single-threaded runtime)
this is after context processing introducing network dependency to the deserialization of data, wasting time and data on non-server cases (e.g. activitypub C2S). sure you can cache individual contexts, but then the context can change underneath you, desynchronizing your cached context and, in the worst case, opening you up to security vulnerabilities
json-ld is not my favorite part of this protocolhm, we really need to differentiate between users responsibility and dev responsibility.
Not sure if Hong saw the draft about the AP kv thing, it supports either JSON-LD fields _or_ as:attachment / as:context …
wtf do I want to say.user story:
We are working on 2 major and 3 projects fulltime which is
- federation of wikibase / wikidata
- federation of Public Broadcasters https://www.publicmediaalliance.org/public-broadcasters-create-public-spaces-incubator/
and these https://codeberg.org/Menschys/fedi-codebaseLet's say we want to federate a Country, then all the knowledge is sent in `attachment` with the fully qualified qikidata url in `context` [as:context - not @context ! - this is so confusing :)]
For example the according entries from the PressFreedomIndex `collection` (co-founder of freelens hereBut anyway, the idea about having
"wd": "https://www.wikidata.org/wiki/Special:EntityData/",
"wdt": "https://www.wikidata.org/prop/direct/" in the `@context` was that any user can consume and federate wikibase
incl.
🧵 1/2 -
hm, we really need to differentiate between users responsibility and dev responsibility.
Not sure if Hong saw the draft about the AP kv thing, it supports either JSON-LD fields _or_ as:attachment / as:context …
wtf do I want to say.user story:
We are working on 2 major and 3 projects fulltime which is
- federation of wikibase / wikidata
- federation of Public Broadcasters https://www.publicmediaalliance.org/public-broadcasters-create-public-spaces-incubator/
and these https://codeberg.org/Menschys/fedi-codebaseLet's say we want to federate a Country, then all the knowledge is sent in `attachment` with the fully qualified qikidata url in `context` [as:context - not @context ! - this is so confusing :)]
For example the according entries from the PressFreedomIndex `collection` (co-founder of freelens hereBut anyway, the idea about having
"wd": "https://www.wikidata.org/wiki/Special:EntityData/",
"wdt": "https://www.wikidata.org/prop/direct/" in the `@context` was that any user can consume and federate wikibase
incl.
🧵 1/2incl.
- the properties in all the languages of the world
- the knowledge of the world in all the languages
- the wikidata relations and qualified statements including the nameMap etc. and all the urls to all wikiprojects incl. their languages and knowledgeHow else could I say to other softwares if they want all users qualified data, use wikidata vocabulary?
wikipedia, wikidata, EBU, Public Broadcasters, taxi data is _all_ JSON-LD …
