? Editing: Post:21.body Save Delete Cancel

Newest topics

Follow in NewsfeedFollowing
+ Start new topic
stickied

Title

Body
^1 ^2 added ━ started by user_name
stickied

The data structures of the major site engines

**ZeroBlog** * data/data.json * post (array) * `post_id` * data/users/*/data.json * comment (array) * `comment_id` * `post_id` * post_vote (hash map) * `"{post_id}"` -> 1 **ZeroTalk** * data/users/*/data.json * topic (array) * `topic_id` * comment (hash map) * `"{topic_id}_{auth_address}"` -> * `comment_id` * topic_vote (hash map) * `"{topic_id}_{auth_address}"` -> 1 * comment_vote (hash map) * `"{comment_id}_{auth_address}"` -> 1 **ZeroMe** * data/users/*/data.json * post (array) * `post_id` * comment (array) * `comment_id` * `post_uri` (`"{auth_address}_post_id"`) * post_like (hash map) * `"{post_uri}"` -> date * follow (array) * `follow_id` * `hub` * `auth_address` Basically, it's all about the same, but the minor differences make the databases incompatible. I have had an idea of implementing an unified engine, that can transparently handle all 3 types of sites, but it seems not to be easily possible with the existing DB design. dbschema doesn't support importing data of varying structure from files of the same name. It is unclear, if the merger feature can solve the issue. At least, I see no evidence of that in the dbscheme of ZeroMe: ``` "db_file": "merged-ZeroMe/ZeroMe.db", "version": 3, "maps": { ".+/data/userdb/.+/content.json": { "to_json_table": [ "cert_auth_type", "cert_user_id" ], "to_table": ["user"] }, ".+/data/userdb/users.*json": { "to_table": ["user"] }, ".+/data/users/.+/content.json": { "to_json_table": [ "cert_auth_type", "cert_user_id" ], "file_name": "data.json" }, ".+/data/users/.+/data.json": { "to_table": [ "post", "comment", "follow", {"node": "post_like", "table": "post_like", "key_col": "post_uri", "val_col": "date_added"} ], "to_json_table": [ "hub", "user_name", "avatar", "intro" ] } }, ``` `".+/data/users/.+/data.json"` has no merger prefix, but perhaps if I add `"merged-ZeroMe"` prefix, it will work. I should check it out. If so, it allows parsing different DB layouts into different tables without conflicts. So... we have **ZeroMe** scheme, which should be supported just because it's the only hub-based media right now. We also have to support **ZeroTalk** scheme to make the transparent migration between vanilla ZeroTalk and our engine possible. But the both are kind of not perfect, so, ideally, we should support the 3rd, redesigned scheme as well. Looks complicated.
^1 ^2 0 comments on Feb 07, 2019 ━ started by geekless
stickied

Features, missing from ZeroTalk engine

TODO: * Moderation: * Ability for the site owner and moderators to modify or delete any posts from UI. * Ability to manage users (ban, unban, modify limits) from UI. * Ability to report a post to a moderator in a safe way (encripted, so only the moderator can see the report). * 4-level moderation system: * Administrator (Site Owner) * Supermoderator (ability to modify `data/users/content.json`) * Moderator (ability to modify posts directly) * Corrector (ability to edit or hide posts, but users can choose if they trust a corrector or not) * 3 levels of stickied posts: by Administrator, by Supermoderator, by Moderator/Corrector. * Ability to split and merge topics. * UI * Enable support for images in the markdown parser. * Post preview. * The name of the last commentator (Now it is not shown: *"1 comment on Jan 29, 2019 ━ started by geekless"*) * User profile pages. * ZeroMe integration. * Garbage collection: ability to easily find and remove comments for deleted topics, votes for deleted comments etc. * The vote counter should start from zero. * The *self*-votes should be ignored. * The vote-based "karma" rating. * Sorting the topic list by "started" date, vote count, comment count, integral activity rating. * Data storage: * Support for hubs. Each forum section should be placed on a different hub. Topics can be migrated between hubs. * Security: * Each entry (post, comment, vote) should be signed individually, so that when a moderator edits and signs a user data, unmodified entries keep their signatures. The security state of each entry should be visible in the UI. (Unsigned votes probably should not be processed at all.) * Other: * All the files of the engine should be moved to a single subfolder, so that the write access for updating the engine can be assigned to a separate set of keys, and shouldn't require the primary site key. [GitCenter/ZeroTalk++](http://127.0.0.1:43110/1H3qtUJRrghDHpY89CBeueVAZw8xbHuDLr/)
http://127.0.0.1:43110/1H3qtUJRrghDHpY89CBeueVAZw8xbHuDLr/
^6 ^7 5 comments last on Feb 06, 2019 ━ started by geekless
stickied

The primary key problem

### Role of the primary key in the functioning of the site What I call the primary private key is a private key, that matches to the site address. If the primary private key is leaked or lost, that is an emergency without solution, and the control over the site is lost forever. For that reason, the primary key should be kept in a way as secure as possible and be used only in absolutely critical situations. In the everyday work, secondary keys should be used, so if they leaked or lost, new secondary keys can be created and signed with the primary key. Unfortunately, the commonly used ways of working with zites now are very careless about the primary key. ### The current status Here are some steps we can take for the better security at FZNC sites: * The moderation should be done using the moderators' own keys, not the primary key. Status: totally possible. We already do that. * Appointment and removal of moderators (and supermoderators) should not require the primary key. Status: not implemented, but looks possible. We need an intermediate `data/content.json`, that lies between the root `content.json` and `data/users/content.json`. * The engine updates should not require the primary key. Status: more investigation is required; the implementation may have some difficulties. All the engine files should be placed in a separate subdirectory, so that a separate `content.json` can be used for signing. What can be a trouble is `index.html`, since his name and location is fixed on the level of the ZeroNet engine. That file should probably be a simple proxy to the real entry point and have no code that can require update. * Changing internal site settings should not require the primary key. Status: not yet implemented. Some engines, including ZeroTalk, make use of the root `content.json` to save settings. Those settings should probably be moved to `data/data.json` and controlled by the intermediate `data/content.json`. * Changing the site metadata should not require the primary key. Status: currently impossible, modification of ZeroNet engine is required. Metadata, such as the site title and description (and maybe some other in the future) are saved in the root `content.json`. We do need the primary key to modify the title, displayed in the "Hello ZeroNet" UI. ### Possible emergencies and their consequences In the case of an ideal separation of tasks of the primary and secondary keys, the following situations and their consequences are possible: | Primary key | Secondary key | Result | | -------------|---------------|---------| | okay | lost | The secondary key is dropped and a new one is created. | | okay | leaked | The secondary key is dropped and a new one is created. The site is restored from a backup, if there was some damage. | | lost | okay | The site keep working. | | lost | lost | The site keep working, but any management and moderation is not possible. | | lost | leaked | The site most likely be destroed; exclusive control over the address is lost. | | leaked | (irrelevant) | The site most likely be destroed; exclusive control over the address is lost. |
^3 ^4 4 comments last on Feb 06, 2019 ━ started by geekless
stickied

(discussion with Kaffie)

> I don't mind providing input, but I don't like the idea. The two big problems outlined appeared to be spam and death of solely managed zites. > > The second problem has already been solved. As there's ways to easily clone a site and move on, and also merger sites which should become the norm. Merger sites allow the backend posting to be decentralized, so there's no single point of failure (just post to a different hub, or have your own). While the front end can be easily cloned and managed if the original developer stops for whatever reason. See: ZeroMe. ZeroMe won't die, even if Nofish goes away. As the hubs are decentralized (no reliance on him adjusting permissions), and the front end can be cloned and managed. > > The first problem, with the spam, is indeed an issue. Having hub providers ban, and users be able to block (and prevent the downloading of a user's posts) are more than enough to stop spam. If need be, switch to a proof of work/stake/identity cert provider. For instance, have a cert provider that requires people to show they are an authentic user and are manually approved. From there we have "trusted certs" which can be allowed, and in the case of a raid, just ban posting from non-trusted cert providers. A variety of solutions could be done here. > > I'd rather not put major zites in the hands of a few. But rather focus on turning them into mergers so that moderation is decentralized. We saw the death of zites like ZeroTV and ZeroTorrent. The dev leaves, and they either don't update, or users eventually run out of space. Merger sites solve this entirely. > > Though I do like the idea of developing a set of core zites that have these decentralization and protection principles in mind. > > What we really need is focus. We have like 3 wikis with split users, and that lead to apathy. There's a ton of forums, but it's hard to find the ones people are talking on. Something like ZeroVoat but with merger sites would probably be ideal for a forum. KopyKate is great, but appears to be suffering from the centralized owner issue. No hubs = not much incentive to post as the next video site will pop up and end up starting from scratch. > > I think a good idea of the council would be to design proper/good merger site standards. So that the same sets of data can be accessed by a variety of sites. Proper post metadata, design structures, etc. That way we don't end up with duplicated work. Why have 3 wikis all dying with different standards, when we could have a single merger db accessed by 3 different wiki frontends? Thank you for the feedback! About hubs - sure, it is the right design, but it's not a silver bullet. Popular sites have a value that grows over time - that is the address. If the address is referenced from a million places, when the site gets unmaintained, it will be the real pain. We faced with that on Ru-ZeroTalk. Its owner, @shift, set the limit of 30K and then disappeared for years. That was a real issue, since the address is referenced from the main ZeroTalk, from ZeroTalk clones in other languages, from blogs, from ZeroWiki etc. So if you want to run an alternative, you have to contact to all the owners of localized ZeroTalks and ask for changing the links in the top bar. Maybe some refuse or already dead or don't go online for years too. Balancer ran a couple of alternative forums, but they weren't too popular, maybe many users just didn't get to them. People just went to Ru-ZeroTalk, were taking for 2 or 3 days, and then disappeared, when limit reached. A few days ago, @shift was back and increased the limit greatly. Suddenly, it turned out we had a great number of active users there, haha. I agree, the hub system is a great improvement. But it still cannot solve that kind of situations in a transparent way. If Ru-ZeroTalk had the hub support, we could run some alternative hubs. Nice. But it is still the problem letting users know they should activate the new hub. The external links still point to the main interface site; there is no posibility to stick the topic containing the notification, etc. There is no big difference: asking users for migrating to another site or asking users for migrating to another hub. It requires some manual actions from a user in the both cases. An unmaintained site also gets no updates of the engine, no matter if it supports hubs or not. (Ru-ZeroTalk seems to run the 2 years old engine.) So, the address issue is still the issue. Actually, I have no idea, how it can be solved in a perfect way. You either control the address or not. If not, the game is over. For that reason, the community should develop principles of the collective management for the decentralized sites. Now zites are managed in the same way how the first pages on the Internet were managed: one person, who does everything. But as the network grows, it won't be enough. Here're my thoughts on the matter: http://127.0.0.1:43110/1fznczNZUMEMvCiqSmCZGUiv5sVnRcsTD/?Topic:1549045300_1GooUE19488nDwG3TdkM8seYAHct4gjkq4/ So (in the best case, when the keys aren't leaked) we can distinguish 3 stages of the zite’s life: 1) the primary key owner is alive and maintains the zite. 2) the primary key owner is dead or the key is lost, but the secondary keys are okay. 3) all the keys are lost, so the zite is completely unmaintained. That is the fate of any cryptography-based site, and what we can do is to somewhat extend their life, cooperating together. A single person may die any day, but if there is some cooperation, together we can guarantee, maybe, 40-50 years of life for a site. Spam. The proof of work is not a solution for the same reason as blockchain is not a solution (of anything, I guess. lol). Corporations can provide a "proof" for a million spam accounts, while an individual tries to mine just a single one. Since Zeronet challenges all the centralized web, big money of big people is at stake. So we should be ready for the worst possible scenarios. PoW can be useful to some extent, while coroporations don't take us seriously. Here's my idea: http://127.0.0.1:43110/1BLoGBTid3NhGu8ts3fAfHJprnbrH3wfTV/?Post:41 But I'm actually not sure if that idea is worth a cent. Proof of identity can be helpful too, but it brings us back to centralized things. The question is "Do you trust the certification center or not?". An answer may be complicated. Probably, you trust, but not absolutely. Probably, we need several independent centers. Probably we want to make sure they are truly independent, but we have no idea how it can be done. The certification center stops functioning someday, and sites should be updated to some new center or centers in order to allow new users access. But many sites are unmaintained... Anyway, all the things we discuss here are nice ideas, that have no implementation. We have no support for hubs in ZeroTalk. We have no support for PoW. We have just one generally accepted certification center, that requires from users no kinds of proof. Etc. A lot of work waits ahead, we now are just scratching the surface of the door to the P2P world. So I just thought, I should do what I could. I know how to run a Clearnet forum. I know how to write code. So I'm going to try running a forum in ZeroNet and make the engine as powerful as possible, compared to popular forum engines for Clearnet. (http://127.0.0.1:43110/1fznczNZUMEMvCiqSmCZGUiv5sVnRcsTD/?Topic:1549014898_1GooUE19488nDwG3TdkM8seYAHct4gjkq4/) The same about the user management. Maybe we'll get some wonderful client-side tools for that, but until that, I just stick to the strategy that the forum should be switched to the whitelisting policy in the case of DoS. So we need some moderators who are technically able to do that (and some scripts). And so on. -------------------- Several side considerations: > See: ZeroMe. Looks like ZeroMe should use the hub-based user registry too. The current registry sticks to the single certificate provider, so the profile seach doesn't work for alternative providers. A user should be able to choose, which registries he/she wants to use. Now it is not possible, since the address of the registry is hardcoded. > But rather focus on turning them into mergers so that moderation is decentralized. It tends to single-user hubs finally. I totally like the idea, and I seemed to already post some thoughts about that in comments in Shouko's Blog: http://127.0.0.1:43110/blog.shouko.bit/?Post:5 With such design, there's no difference between a single-user hub and a blog, and there's little or no difference between a social network front-end and ZeroHello news feed. On the other hand, we can succesfully use more centralized engines as well, depending on the purpose. When you post a message on a site that is controoled not by you, the owner can do with your data just anything. He/she can modify it, delete it, create some posts that you didn't actually write etc. The existing engines hides that possibility and gives you the false impression that your data is safe. But it is just a lack of interface for taking moderation actions. Technically, it is totally possible and is supported by ZeroNet engine, so the site owner can modify your files with a text editor right now. I believe, we should try all possible ways and design choices. If we are able to create a decentralized (in terms of the data storage) forum with the centralized moderation system, we should try and see what it can be useful for. If we can create a completely decentralized social media, composed from thousands of hubs, we should try it too. That is why both centralized moderation and decentralized hubs are planned in my ZeroTalk todo list. > I'd rather not put major zites in the hands of a few. As I said above, the infrastructure depends on cryptographic addresses, and addresses are the major point of failure. We can potentially make a new fork of the front-end every week, but it doesn't address the issue, how to switch to new addresses and how we can determine if we can trust them. Get me right, it's not that I'm trying to control major components of the network. I run some sites and invite people to take a part in their management. Why? Because a team can reach better results than an individual. There are many successful forums with a great community on Clearnet, that are maintained by enthusiasts, and we already know how to manage that kind of things. I'm not talking about commercial projects, but about such projects as for example lingvoforum.ru, where you can contact to the admin, and he is a real person, who is running his forum for his own money for 15+ years. Sure, we should develop better tools, but we need to start somewhere.
http://127.0.0.1:43110/1fznczNZUMEMvCiqSmCZGUiv5sVnRcsTD/?Topic:1549045300_1GooUE19488nDwG3TdkM8seYAHct4gjkq4/
^3 ^4 9 comments last on Feb 06, 2019 ━ started by geekless
stickied

Welcome to the <Talks: Council's Services Support and Development> forum of the The First ZeroNet Council!

* [The Forum Rules](http://127.0.0.1:43110/1fzncGxeyDpbCz1CkvBkooEDsXGFq1GfK/?Topic:1549008843_1GooUE19488nDwG3TdkM8seYAHct4gjkq4/) * [Why we need the ZeroNet Council](http://127.0.0.1:43110/1fzncZRMPGPBdyUBm2ATVSkRAsnvMq5VN/?Post:2) * [The Manifest of The First ZeroNet Council](http://127.0.0.1:43110/1fzncZRMPGPBdyUBm2ATVSkRAsnvMq5VN/?Post:1)
http://127.0.0.1:43110/1fzncGxeyDpbCz1CkvBkooEDsXGFq1GfK/?Topic:1549008843_1GooUE19488nDwG3TdkM8seYAHct4gjkq4/
^2 ^3 0 comments on Jan 29, 2019 ━ started by geekless

 

Follow in NewsfeedFollowing

Title

Body
^? ^0 username posted added
Please sign innew comment
Sign in as...
Submit comment
You are running out of your allowed space, please contact the site's admin at unknown to raise your limit.
user_nameadded ^1 ^2
Reply
Body
More comments
This page is a preview of ZeroNet. Start your own ZeroNet for complete experience. Learn More