Nginx how to; Url cleaning - removing multiple slashes

10 Jan 2010

Update: 2012

This functionality is now natively available in Nginx see: merge slashes. I'm not sure that will issue a 301 redirect, so the SEO implications may still be vaild.


Carrying on with my short posts on configuring Nginx, heres an easy tip to ensure that all urls are clean, multiple slashes are removed and redirected to the cleaned url eg: http://example.com//search/ and will be redirected to: http://example.com/search/.

**Updated Sept 15th

# Remove any multislashes in the url
  # $uri is a cleaned version of the url
  # so we test against the $requested_uri
  set $clean_uri        $uri$is_args$args;
  if ( $clean_uri != $request_uri ) {
    rewrite ^/(.*)      $scheme://$host/$1    permanent;
  }

This will stop you serving the same content on multiple urls which has SEO implications. In the past I have written middleware to do the check and redirects, but as the Nginx Proxy Module uses the cleaned $uri the middleware never sees the multiples slashes and doesn't know without looking at X-Headers that it is serving an invalid url.

* Wondering why the redirect doesn't redirect to itself? The regular expression just captures everything after the first slash! Well the rewrite module also uses the cleaned $uri!

** Update: June 24th

It seems that you can get into a nasty redirect loop, so if your just cleaning double slashes you can use this:

# Remove any multislashes in the url
    if ($request_uri ~* "\/\/") {
      rewrite ^/(.*)      $scheme://$host/$1    permanent;
    }

 

** Update: Sept 15th 2010

Jon Topper found a bug with the previous fix - http://www.example.com//?// would cause a redirect loop. He posted in the comments a fix and I'm putting here so its easier to read. Thanks Jon.

# Remove any multislashes in the url
    # Tested against:
    #   http://www.example.com//
    #   http://www.example.com/w//w//w//
    #   http://www.example.com//?//
    set $test_uri $scheme://$host$request_uri;
    if ($test_uri != $scheme://$host$uri$is_args$args) {
        rewrite ^ $scheme://$host$uri$is_args$args? permanent;
    }