WelcomeWEFreedomWindsorEssexCounty-ActionCommunicationsRalliesResourcesSocialEventsWednesdayNightMeetings-ActivismAgoraAgorismCanadaCommonLawCommunityDataCentersDataCentresDoocracyFarmersMarketIntroductionsNaturalLawNewslettersOntarioOrganizingOutreachProjexProjexWikiRalliesResistanceAnalysisSoireesSoiréesWikiSpooksWindsorElectionsTop Subs
4
MatrixDev - - (+3|-0)
The front page will always have a full page of content even if it has cross posts to deduplicate. (self)
7 commentsbyx0x7

As you guys know the submit form lets you post to more than one sub at a time by separating them with commas. This is encouraged because we want to fill out both generic and more specific subs.

But this means that there would be a lot of repeat content on the front page if people use this feature (which we want them to). So to reduce the consequences of that the front page de-duplicates posts based on the URL. A side consequence of that solution is sometimes the front page is shorter than a full page.

This is no longer a problem. I have a library I wrote called matrixdb that is a wrapper to a relational database. It basically makes a vector database (I've been running this longer than those have been popular). That library now supports a dedup option. But what's best is it always returns the correct number of items queried for even when deduplicating.

Let me know if you see any bugs.

Comment preview

[-]JasonCarswell1(+1|0)

Any chance we could see this kind of a feature (no rush) at page footers (comments, messages, posts, etc.)? Copied from WikiSpooks:

View (previous 1,000 | next 1,000) (20 | 50 | 100 | 250 | 500)

There were only 895 items in that list so I'm not sure why it even says "previous".

It says 1000 because in the URL you can change "500" to any number, those are just the defaults offered.

It would also be good to list the page number - "Page 7" or even better, "Page 7 of 9" if it's practical and not too burdensome to compute like "Page 5432911 of 696969696969".

parentlinkreply
[-]x0x70(0|0)

It does take more compute to display 1000 items in a constantly re-sorting list whose sort is custom for each user. Reddit doesn't do it for a reason. If someone really wants to dig for posts it's a good invitation for them to use subs.

parentlinkreply
[-]LarrySwinger0(0|0)

Cool. Why not dedup every page btw including profiles?

parentlinkreply
[-]x0x70(0|0)

Now that it's just an option passed to the "database" it is more viable to do that because I don't need to have custom code everywhere.

My argument for not deduplicating is there needs to be a way to find all of the posts. If I deduplicate in 100% of cases some of these cross posts basically don't exist. Then why even have their row in the database?

So because posts should be find-able I think I should do close to zero deduplication, except where not doing so creates a high annoyance on a high trafficked page. Which basically means just the front page. I think the more you get off the front page the more people should see just plain data.

parentlinkreply
[-]LarrySwinger0(0|0)

Why would it be any less of an annoyance on a profile page than on the frontpage? To find the x-posts there's a link "other discussions" (or w/e it's called). Maybe fewer people are annoyed if there's no dedup on profile pages but the annoyance itself is equally big. I don't see why any page except the "other discussions" page should be protected from dedup.

To make it easier to find the other discussions, the "other discussions" link can be placed in the overview maybe on the right hand side. The cost is that those menus (where it says the title, x comments, the username, and tag) starts to become cluttered, but otherwise it's an extra click to find the other discussions because they have to visit a post's permalink. So that's a dilemma but I think they're both better options.

parentlinkreply
[-]x0x70(0|0)

I think one way to look at it is the intent behind looking at one thing or another. A person looking at the front page has the intent of seeing novel content. A person looking at a user profile has the intent of seeing everything a user has posted.

I also think the more a page is viewed the more like a curated experience it should be, and the more specifying information a user gives for what they want to see the more it should behave like a database. A database isn't going to deduplicate things. It's going to give you everything raw. But the front page is an exception to that principle.

parentlinkreply
[-]LarrySwinger0(0|0)

I guess I just prefer tags where multiple are assigned to a single post. Separate x-posts makes more sense for big sites where each sub will have comments coming out of its own culture instead. That made sense for Reddit since it was planned as a large scale things they had bots to get it off the ground and investment from YC with which they probably paid for advertisements as well. This is an organically growing site where most x-posts will have 0 comments so seeing multiple even on a profile page is just unnecessary clutter just my 2c we can agree to disagree.

parentlinkreply