Recently I had an idea for a research tool, I realized I'd need to be able to find websites programmatically and the webs data is behind the largest corporations, not something we reliably have access to for being able to develop around.

So it got me thinking, how lightweight could you first build a crawler if you only cared about getting meta data (titles, description and the url)? Then it got me thinking, could we not share the load, create a distributed peer to peer network and make this data decentralized?

So what I have built is the open web directory. A network that crawls and builds a yellow pages like directory for web pages online.

You can setup your own node and contribute to the network by simply running:

git clone https://github.com/idev-games/the-open-web-directory.git cd the-open-web-directory npm start 

And open port 80 for that device by port forwarding.

I think this is going to be a fascinating experiment, it's early days so expect bugs and issues but give it ago and let's see if we can index the web.

Check out more info here:

https://github.com/iDev-Games/The-Open-Web-Directory

The site listed on github is the live front end (you can host the html files in the public folder anywhere to open another frontend on the web).

Hello, I have been thinking about a project I‘d like to try out, where I have like a Database, an api server and a web server. It‘s not too large, and mostly to learn about how they all interact. The issue is, I would prefer not having to pay for multiple servers to host everything. Is there a way to simulate how it would be in real life on a single machine?

Hey guys, just after opinions really. If you’re launching a somewhat large site, with quite a bit of stuff going on, would you rely on your own testing or would you suggest getting a third party tester? It’s mostly for things like UX I’m wanting to test. I know have I would expect things to work but being the sole developer I think I may have gone a bit blind to somethings. I have friends do bits of pieces in terms of checking things but I wonder if the consensus would be to just get an actual tester to go through everything, testing the functionality all works, different variations, ensure the UX is as good as it can be etc or do you think I’m overthinking it?

so im working on my final year project report (pfe) (projet fin d'annee = Final year project which is literlly a big project to final my 6 years, in uni ) and i had to do the whole uml thing, use case diagram, class diagram, sequence diagrams for the main flows

and while i was doing it i kept asking myself, how do i actually know if what im drawing is "correct"

like the global logic of the app is obviously fixed, i know what the app does and how it should work, but when it comes to modeling it visually there are like ten different ways to represent the same thing. someone else looking at the same requirements could come up with a completely different class breakdown or a different way of grouping use cases, and technically neither version would be "wrong"

so when the jury reviews it, are they checking for one specific correct answer, or are they more just checking if your model is internally consistent and reflects a reasonable understanding of the system

anyone who's been through something similar, academic reviews, code reviews, whatever, how do you even validate a diagram when there's no single ground truth to compare it against

side note for context since some people outside the french education system might not know this, the PFE report is usually structured around sprints, like sprint 0 for environment setup, then sprint 1, 2, 3 etc each with its own objective and backlog, and for every single sprint you're expected to redo a use case diagram, a class diagram and sometimes sequence diagrams specific to that sprint

i pulled up an example report just to check and it had 6 sprints and something like 57 figures total, most of them being use case, class or sequence diagrams repeated per sprint. so its not like you draw one diagram for the whole app, you basically draw a mini version of the same three diagrams every sprint, which is part of why i started wondering how strict the "correctness" of each one actually needs to be

Does anyone know how to add completely secure video calls to a website? I’m struggling to find the best way.

Additionally, how do I then host this for free / as low cost as possible.

If anyone has advice that would be great!

Which hosting platform would you pick for a small production app: Render vs Railway vs Cloudflare Workers?

I’m building something like a inventory management app and website — authenticated users, a Node/TypeScript API, PostgreSQL, background jobs/cron, and potentially a few thousand users over time.

Currently considering **Render, Railway, and Cloudflare Workers**.

For a production app where reliability, scaling, DB latency, and predictable costs matter, which one would you choose and why?

Would you recommend something else like **Fly.io, AWS, or Vercel + another backend** instead?

Curious to hear from people who’ve actually run production workloads on these.

I'm looking for a workable solution or recommendations to provide a client's credentials to that client in the event of my untimely demise or absence. (E.g. hit by a bus - falling out and need to hand everything over, etc.) I also don't want to mix MY credentials with theirs.

So my primary work isn't webdev, but I do some for clients as part of my general services. Generally this means me building everything from scratch and leaving the client(s) out of the day-to-day management entirely.

I'm trying to figure out the best way to have their business' credentials saved, updated as they are changed, and set aside for if I get "hit by a bus" so they won't be locked out of everything.

I also don't want them "futzing" with things (because some of these clients have impulse control issues and would 100% give their credentials to someone on Fiver to update a paragraph on their website at 2am when I don't answer my phone.) So access while I'm not dead or fired isn't a real option.

I understand that I can make "organizations" in some of the password managers and have emergency hand-over to pre-selected people, but I work alone, and don't "have a lawyer" that can manage it for me.

A feature common to a great very many HTML boilerplates is this line:

html <meta name="viewport" content="width=device-width, initial-scale=1.0">

The initial-scale=1.0 bit was present to fix an ancient issue on iPhones. When rotating the screen into a landscape orientation, the page would keep the same width as when it was in portrait, making everything appear very zoomed in.

This isn't an issue any more. I've tested it across very many devices, and it appears it was fixed in iOS 9, which released back in 2015. In addition to testing, I've consulted a great number of other professionals, and nobody has been able to find evidence of it being needed any longer.

You can safely simplify the meta viewport declaration down to:

html <meta name="viewport" content="width=device-width">

It isn't hurting you to leave it there, but it is vestigial and unnecessary.

Further details: - https://vale.rocks/micros/20260902-1350 - https://quirksmode.org/quirksblog/2026/0902-initial.html

I am considering building a family website for my family. My mother's extended family built one, but it's something nobody really ever visits. I'm considering building a wiki-style site (locked behind user accounts) to have a page for each family member, each nuclear family, and then they can build upon it and create pages for vacations and things like that. I would then have a mini social feed for us each to share pages we've created. For example, my sister might say "Just made a page for our Disney Land 2026 trip! Here's the link."

Might be overscoped, but I thought it'd be a fun way to start capturing family stories.

If you've built a family site for your family (or someone else's) what did it have? What purpose did it serve and what features were built into it?

Let’s share hidden gem tools, not the usual stuff that shows up in every "top developer tools" post, but smaller tools and projects that made you think, "wait, this is actually really useful."

I'll start with 3 I've come across:

1. Archify
https://github.com/tt-a1i/archify

Found this recently. It's an agent skill that works with Claude Code, Cursor and Codex and basically turns a codebase into an interactive architecture map.

You can export it as HTML/PNG/SVG/WebM, and one thing I thought was pretty cool is that you can diff the architecture before a merge to see how a PR changes the system.

2. boneyard-js
https://boneyard.vercel.app

Generates skeleton screens from your actual UI instead of making you manually recreate the layout with placeholder components.

Pretty nice alternative to manually maintaining react-loading-skeleton stuff.

3. Gradient Studio
https://gradientsaas.blogspot.com

Simple procedural CSS gradient generator.

Has stuff like mesh gradients, aurora effects and grain. You mess with the settings and copy the result as CSS, Tailwind or SCSS.

I've been using it mostly for quick landing page backgrounds.

What's one tool you keep recommending that somehow still isn't very well known?

Due to a growing influx of questions on this topic, it has been decided to commit a monthly thread dedicated to this topic to reduce the number of repeat posts on this topic. These types of posts will no longer be allowed in the main thread.

Many of these questions are also addressed in the sub FAQ or may have been asked in previous monthly career threads.

Subs dedicated to these types of questions include r/cscareerquestions for general and opened ended career questions and r/learnprogramming for early learning questions.

A general recommendation of topics to learn to become industry ready include:

• HTML/CSS/JS Bootcamp
• Version control
• Automation
• Front End Frameworks (React/Vue/Etc)
• APIs and CRUD
• Testing (Unit and Integration)
• Common Design Patterns

You will also need a portfolio of work with 4-5 personal projects you built, and a resume/CV to apply for work.

Plan for 6-12 months of self study and project production for your portfolio before applying for work.

My main domain is 25+ years old and has a great reputation, never used for spam or anything like that. I've been having the emails forwarded through WHM / cPanel to my Gmail.

7 days ago I moved my main site to a new VPS, and copied over the forwarders. But today I discovered that I'm missing the wide majority of my emails! I found them in WHM > View Relayers, about 241 out of 250 were rejected by my Gmail with this error:

TLS_AES_256_GCM_SHA384:256 CV=yes : SMTP error from remote mail server after end of data: 550-5.7.1 [<new VPS IP> 19] Gmail has detected that this message is likely\n 550-5.7.1 suspicious due to the very low reputation of the sending domain. To\n 550-5.7.1 best protect our users from spam, the message has been blocked. For\n 550-5.7.1 more information, go to\n 550 5.7.1 https://support.google.com/mail/answer/188131 af79cd13be357-93917765ba8si962882385a.175 - gsmtp 

I double checked in Postmaster Tools, and the domain's reputation is stellar. So when it says "low reputation of the sending domain", I have to assume it means the IP of the new VPS. MXToolbox confirms that the IP isn't on any blacklists, although the parent ASN is on the UCEPROTECT-Level3 list (which covers thousands of domains so it's not MY domain, but I can pay an extortion fee to get mine whitelisted).

Any suggestions?

With AI crawlers and agents hitting websites constantly, how much of what analytics calls “direct traffic” is even human anymore?

If bots are getting counted as visits, conversion rates automatically look worse. And then decisions about landing pages, content, ad spend etc. are being made from numbers with a messed up denominator.

Blocking bots doesn't really solve it either. Some are obviously junk, but others are search crawlers, monitoring tools or AI agents that might actually send users your way.

Feels like analytics needs to get much better at telling why something is accessing a site, not just whether it looks like a bot.

Anyone seeing this noticeably mess with their analytics yet?