Understanding Dynamic Rendering: Why It Matters for Your Website's SEO
When it comes to building a website, the three key components are HTML, CSS, and Javascript. In recent years, client-side rendering (CSR) has become increasingly popular. With this approach, the browser downloads an empty HTML shell and uses Javascript to generate the content on the client side.
[Image credits: web.dev ]
However, while CSR may work well for users, it creates a significant problem for search engines and other bots. Bots rely on the generated HTML content to gather data and index your website. But when a bot visits a client-side rendered page, there is no data available because the Javascript hasn't been executed yet. As a result, the bot only sees an empty website, which can severely impact your website's SEO.
That's where dynamic rendering comes in. Dynamic rendering is the process of rendering a fully formed HTML page on the server side and sending it to bots, while still using client-side rendering for users. This approach allows bots to access the fully rendered HTML content and gather the data they need to index your website, without affecting the user experience.
In this article, we'll explore dynamic rendering in more detail and show you how to build a dynamic renderer using Golang, a powerful and efficient programming language, from scratch. With this knowledge, you'll be able to create high-performance web applications that not only deliver a great user experience but also rank well in search engines.
But do I even need Dynamic Rendering?
Well, that depends.
❌ You don't need dynamic rendering when -
If you are working with meta frameworks like NextJS, NuxtJS, Remix, etc. Because these are server-side rendering applications.
If your codebase is small and you can migrate it to server-side rendering applications
✅ You need dynamic rendering when
- When your codebase is huge and you don't have the bandwidth to migrate it to server-side rendering applications. Also, you want to improve your website as soon as possible.
But keep in mind,
Dynamic rendering is a workaround and not a long-term fix for problems with javascript.
Implementing Dynamic Rendering
Let's first discuss the high-level overview, before jumping into the code.
Since we need to render the HTML before sending it to the bot, we need a server.
We will define a middleware, it intercepts the request made to the web app.
We will use the user agent to check if the request was made by a bot or a user.
If the request was made by a user, we send the client-side rendered app.
But if the request was made by a bot, we generate the HTML content (with something like puppeteer) and then send the rendered HTML to the bot
Now that we are done with the high-level overview, let's jump into the implementation
Setting up server
For setting up the server, I will be using the Gin framework.
func main() {
r := gin.Default()
r.Use(our_middleware)
r.Static("/", "./frontend/dist")
if err := r.Run(":3000"); err != nil {
log.Fatal(err)
}
}
This starts our server at port 3000. Our CSR frontend files are present in the folder frontend/dist. Now when someone sends a request for a page, we serve HTML, CSS, and JS from this folder.
Now we need middleware to intercept the traffic. So let's implement it.
Adding a middleware
func dynamicRenderer() gin.HandlerFunc {
return func(c *gin.Context) {
isBot := checkforBot()
if isBot {
return
}
c.Next()
}
}
func main() {
r := gin.Default()
r.Use(dynamicRenderer())
r.Static("/", "./frontend/dist")
if err := r.Run(":3000"); err != nil {
log.Fatal(err)
}
}
Now that the middleware is set up, let's write the code for rendering the HTML file. But before that, we need to run our front end on some other port. Because the puppeteer scrapes data from the website. So let's write code to start the frontend
Serving the actual front end from another port
func main() {
cmd := exec.Command("command","to","serve","your","frontend")
cmd.Dir = "./frontend"
cmd.Stderr = os.Stderr
wait_for_files_to_be_served()
if err := r.Run(":3000"); err != nil {
log.Fatal(err)
}
}
Writing the code for the renderer
func dynamicRenderer() gin.HandlerFunc {
return func(c *gin.Context) {
isBot := checkForBot()
if isBot {
ctx, cancel := chromedp.NewContext(context.Background())
defer cancel()
url := "http://localhost:" + reactPort + c.Request.URL.Path
var html string
err := chromedp.Run(ctx,
chromedp.Navigate(url),
chromedp.InnerHTML("html", &html, chromedp.NodeVisible, chromedp.ByQuery),
)
if err != nil {
log.Println(err)
c.AbortWithStatus(http.StatusInternalServerError)
return
}
c.Data(http.StatusOK, "text/html; charset=utf-8", []byte(html))
c.AbortWithStatus(http.StatusOK)
}
c.Next()
}
}
Here's a short explanation of what happening:-
Once we know it's a bot, we connect to the puppeteer.
We then navigate to the URL, using chromedp.Navigate(url) .
We wait for all the children of the HTML tag to load by using chromedp.NodeVisible
Once the javascript has generated all the HTML content we store it in html variable and send it to the bot.
Well, that's the entire implementation !! (sort of). I have left some boring parts out, but if you want you can check this repository.
Now, let's look at the result
- Our webpage looks like this
- When the user is requesting for a file, we get
- But when a bot requests a file, we get
Notice that the HTML is all rendered.
NOTE: When we send a rendered page there is not JS. So it won't function. But for a bot that is not an issuse because it does not use JS in any way
Well, that's all. Thanks for reading. If you have any doubts, you can post them it comments. I'll try my best to clear your doubts
