sNews Forum

sNews 1.6 (previous version) => Mods/addons => Topic started by: Joost on July 26, 2008, 11:45:59 pm

Title: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Joost on July 26, 2008, 11:45:59 pm
Thanks to funlw65 (http://snewscms.com/forum/index.php?action=profile;u=7634) the code is updated

sNews shows identical content regardless whether a requested url ends with a slash or not.
If this requests returns content:
http;//booptrap.com/category/
This one returns the same:
http;//booptrap.com/category
So whenever a site links to an sNews driven site, omiting the trailing slash, Google will probably mark that as duplicate content.

Using .htaccess it is possible to let sNews redirect(permanent, 301) urls without trailing slashes.

Here is the rewrite rule replacement for .htaccess. You will have to edit #RewriteBase /sNews16 as usual.

Quote
RewriteEngine On
#RewriteBase /sNews16
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^(.+) $1 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.+)/$ index.php?category=$1 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.+)$ $1/ [R=301,L]


... And here is the old version, which simply returns a "Page not found message"

Quote
RewriteEngine On
#RewriteBase /sNews16
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^(.+) $1 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.+)/$ index.php?category=$1 [L]
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: funlw65 on July 27, 2008, 12:30:39 am
Oho Joost!
I just wanted to post  a reply in Suggestions forums, to support Sven's request.
Thank you very much, K++
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Sven on July 27, 2008, 09:38:14 am
Hi fellows.
As usual, I don't get it at all.
What should be done exactly. ???
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: funlw65 on July 27, 2008, 09:54:44 am
Well, for now, the link

http;//sitename.com/category
will direct you to a 404
and http;//sitename.com/category/ will work as expected
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Sven on July 27, 2008, 10:04:58 am
 ???

RewriteEngine On
#RewriteBase /sNews16
mine is:
Quote
RewriteEngine On
#
RewriteBase /
RewriteCond %{REQUEST_FILENAME} -f
(...)
Quote
#RewriteBase /database
gives an internal error (500).
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: funlw65 on July 27, 2008, 10:13:30 am
And if you let it at it is?
It counts only what is after it because you don't have snews under a sub-folder....

edit:
RewriteBase can remain as it is on your file
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Sven on July 27, 2008, 10:24:41 am
It counts only what is after it because you don't have snews under a sub-folder....
okey. Got it now. Thanks pal. ;)
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: funlw65 on July 27, 2008, 05:05:02 pm
Joost,
What about this (http://www.mydigitallife.info/2007/03/19/add-trailing-slash-to-the-end-of-the-url-with-htaccess-rewrite-rules/)?

Links without trailing slashes will be redirected (301) to links with trailing slashes. Of course, this will affect tag words which don't need trailing slash at the end but maybe this can be resolved inside code, not looked yet into...
I think tag case can be managed as in examples from that page....

What you think?

------------------------
EDIT:
The next rule for your already written rules can be this?
RewriteRule ^(.*)$ $1/ [L,R=301]
This way, all links without trailing slash will be redirect to one with trailing slash ... is working on my localhost but I'm not expert....

If this can meet your approval, can be:
RewriteEngine On
#RewriteBase /sNews16
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^(.+) $1 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.+)/$ index.php?category=$1 [L]
RewriteRule ^(.+)$ $1/ [L,R=301]

Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Joost on July 27, 2008, 06:55:47 pm
I am not an expert either. However, logic and common sense, usual is my salvation(amen) ;).
It seems we both have different server configurations.
I adjusted your code, (which is by the way, far better then the code I've tried earlier today) to this:

RewriteEngine On
#RewriteBase /sNews16
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^(.+) $1 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.+)/$ index.php?category=$1 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.+)$ $1/ [R=301,L]

I needed the extra line, to prevent .htaccess from adding an extra slash. The red line tells the server only to add slashes if it is not a directory.
I've tested yours and mine like this:
I've created a testmap, containing an index.html and somefile.html. I also have a somefile.html in the same directory as snews. That way, I am able to test all kinds of request like:

testmap/
testmap
testmap/somefile.html
somefile.html

Maybe you can do some similar testing on your local install?
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: funlw65 on July 27, 2008, 07:12:57 pm
Is working also with that red line on my localhost
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Joost on July 27, 2008, 07:28:01 pm
Is working also with that red line on my localhost

Well done funlw65. :)

Quote from: Nasty forum
An Error Has Occurred!
Sorry, you can't repeat a karma action without waiting 1 hours.
:'(
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: funlw65 on July 27, 2008, 07:41:24 pm
 ;D Thank you Joost, is ok with only one :)
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Sven on July 28, 2008, 06:34:00 pm
Hello Big Indians
you're talking Apache to me. ???
Now my .htaccess is very so diffrent:
Quote
RewriteBase /
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^(.*) $1 [L]
#RewriteCond %{REQUEST_FILENAME} !-d
#RewriteRule ^(.*)/$ $1 [R=301,QSA,L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule tag/(.*)$ index.php?search=$1 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php?category=$1 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([a-z_]+)/([^/]+) index.php?category=$1&title=$2 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([a-z_]+)/([^/]+)/([^/]+)/ index.php?category=$1&title=$2&commentspage=$3 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([a-z0-9_-]+)/([0-9]+)/ index.php?category=$1 articlespage=$2 [L]
Do I have to change something? :P
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Joost on July 28, 2008, 08:03:01 pm
Do I have to change something? :P

I am not sure what code your showing.  ??? It might include modified code to you need to run the site.
Anyway, the code that's published in the first post is the full monty. It meant to replace the .htaccess that's running a default installation.
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Sven on July 29, 2008, 07:39:23 am
Bloody Alzeimer! I can't remember when and why I changed it! :-\
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Armen on July 29, 2008, 03:27:38 pm
Here's the solution I'm using (it's sitewise):

Just after:
Code: [Select]
#RewriteBase /PATH
Add
Code: [Select]
RewriteCond %{REQUEST_URI} ^/[^\.]+[^/]$
RewriteRule ^(.*)$ http://%{HTTP_HOST}/$1/ [R=301,L]

And ALL pages and categories will recieve a trailing slash if none provided.

Joost, thank you again for bringing up another important issue.

It should be added into the default package.
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Sven on July 29, 2008, 03:44:22 pm
Waiter, please?
1 karma for the ogre and for all.
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Joost on July 29, 2008, 05:02:15 pm
Armen,

Your code doesn't work great on my localhost with the additional tests (http://snewscms.com/forum/index.php?topic=7570.msg53973#msg53973).
An unslashed, real subdirectory does get a slash, but instead of redirecting to
http://localhost/sNews16/testmap/
it redirects to
http://localhost/testmap/

I added your code like this:

RewriteEngine On
RewriteBase /sNews16
RewriteCond %{REQUEST_URI} ^/[^\.]+[^/]$
RewriteRule ^(.*)$ http://%{HTTP_HOST}/$1/ [R=301,L]
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^(.*) $1 [L]
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Armen on August 01, 2008, 12:34:33 pm
Hm. No problems on my side, Joost. BTW, I didn't apply the fix from your first post. Just those lines I specified.

Here's my final solution, + removes "index.php" from url if specified):

Code: [Select]

#RewriteBase /
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index\.php\ HTTP/
RewriteRule ^index\.php$ http://%{HTTP_HOST}/ [R=301,L]
RewriteCond %{REQUEST_URI} ^/[^\.]+[^/]$
RewriteRule ^(.*)$ http://%{HTTP_HOST}/$1/ [R=301,L]

RewriteCond %{REQUEST_FILENAME} -f..........
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Sven on August 01, 2008, 01:46:35 pm
No problem for me: on line it works rela fine.
And another karma for my ogre who has thought of the index.php question.
Bravo!
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Ken Dahlin on August 01, 2008, 04:42:39 pm
You know, I honestly didn't think this was a big deal until I checked google page ranks for certain URL with and without trailing slash... and to my shock found that the ones indexed without the slash were PR 0 compared to the same pages with slashes at PR 4, 5 or 6!

Sadly, this also may explain some inconsistent performance in indexing sites with my dynamic sitemap generator mod which does not use trailing slashes in the URIs. My guess is that before I added sitemaps to my site, google had already indexed the site and now sees the sitemap as duplicate content!

Thanks for bringing this to my attention.

Ken
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Keyrocks on August 01, 2008, 06:45:12 pm
@ Joost, Ken and Armen... would this be another "Bug" that should be posted in the "Bugs" board and then patched into the official 1.6 download?
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Ken Dahlin on August 01, 2008, 09:38:42 pm
@ Joost, Ken and Armen... would this be another "Bug" that should be posted in the "Bugs" board and then patched into the official 1.6 download?

That it's a "bug" is debatable as sNews probably still works as intended. But the project claims that "Search Engines dig sNews. Really." and so I'd say this little htaccess rule is a step in the right direction for the official release.
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Armen on August 01, 2008, 10:31:17 pm
Yes, Keyrocks. I think, it should be packed into the default package as soon as possible, because sNews is being promoted as a SEO-friendly CMS.

With latest .htaccess fixes applied, - yes, - very-very friendly.
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Joost on August 02, 2008, 02:08:33 am
Lets not get carried away here.
There are two options presented in the first post.
The oldest code (second) is strictly SE friendly:
In sNews all uris end with a slash. If no slash is appended, Google and visitors get a 404 message.

The new code (Armen's code didn't work in my config and therefore disregarded for now), is more or less user friendly:
If no slash is found at the end of a uri, a slash is appended and redirects to the right location.
However, this approach assumes that the the uri actually exists. If it doesn't exist, it invokes a cascaded response, something like:
- HTTP/1.x 301 Moved Permanently (new location, slash added)
- HTTP/1.x 302 Found ( new location 404/)
- HTTP/1.x 404 Not Found

I am not sure how Google handles this. For that matter I opt for the second.
ps: Of course, the newst code might be convenient, when you have an important (but misspelled)  inbound link.
That, you can repair with a specific redirect rule.
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Armen on August 02, 2008, 09:40:50 am
Regarding errors and "cascaded responses"

You mentioned something about a redirection to 404/ page?

I've already made sNews send "NOT FOUND" headers without redirecting user to "404/" page and thus without changing URL. Default sNews' method (I mean a 403 redirect to 404/) is not SEO-friendly at all. Because in case search engine stumbles upon an inexistent page it recieves "FOUND" http header, then suddenly "403 REDIRECT" and then "404 NOT FOUND".

That's abnormal. Google doesn't like this, because 404 header must be sent at the first place. Some time ago in the webmaster.google.com lounge I've recieved multiple error reports that the engine returns "FOUND" when there must be a "NOT FOUND" header. It cost me some time, but I made the CMS recognize inexistent pages and immediately send a "404" header.

But even when I downloaded and used vanilla 1.6 version of sNews, I still recieve no errors. No "looping" whatsoever, no unlimited redirects, no wrong redirects. Tried everything and still can't find any fault in the rules.

And, BTW, Google knows about No-Slashed-to-Slashed URL 403 redirect. It's considered normal to use this method. It's safe.

Quote
In sNews all uris end with a slash. If no slash is appended, Google and visitors get a 404 message.

Why should one recieve a 404 error, when he can be redirected to the right URL? That's not human-friendly for sure.

Update: BTW, tried your updated code and whenever I call a category without a trailing slash, it gives me a 404 message. Is that how it's supposed to be? I hope, not.

RULE-TEST

You can test my solution here: http://sapehelp.ru/

Try http://sapehelp.ru/money/ and http://sapehelp.ru/money

You'll be conviniently redirected. Try http://sapehelp.ru/index.php and you'll recieve http://sapehelp.ru/

(If you want to see modified 404 engine try anything else, like http://sapehelp.ru/sfewf/ or even http://sapehelp.ru/sfewf (without slash). See? No loops whatsoever...)

Feel free to try out my solution and say what you think.

Another mystery from your first post:

Quote
You will have to edit #RewriteBase /sNews16 as usual.

Why edit a comment?  :D  :D  :D

Why? It's a commented line, FGSake.

BTW, everything works great even without specifying "RewriteBase", when rules are well-written. At least with sNews.
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Joost on August 02, 2008, 07:42:28 pm
There seems to be some misunderstanding here.
- Your code works on your system. However, not on mine. We are looking for (core)code that can live on the majority of servers. If code from the first post behaves badly (403, 500 headers or 404 on existing directories etc), then that code should be tossed out. For me there is no competence issue. It is about server configuration.
FYI: Your code invokes a redirect before a 404 header on http://sapehelp.ru/sfewf as well.

- There is a difference between Apache error handling and sNews (php) error handling:
 Apache can dtermine whether a file or directory exists. But it does not know wheter a database generated page exists.
  Before appending a slash and redirect, Apache has verified the existence of that directory. This is similar to the way domainnames are handled: A slash is appended only if the domainname exists (there's some logic here).
 sNews can determine whether a database generated page exists. To do so, it must work toghether with Apache. The htaccess file tells Apache: "If no such file exists, let sNews handlle it".
That's when the cascaded error handling starts. Cooperation of the two is not flawless and sNews suffers from a bug.
Bug description and fix:
Remove /404/
So unless this fix is applied, a typo like http;//site.com/categor.. instead of http;//site.com/category/
will invoke a cascaded error handling.


 
Quote
You will have to edit #RewriteBase /sNews16 as usual.

Why edit a comment?  :D  :D  :D

Why? It's a commented line, FGSake.
BTW, everything works great even without specifying "RewriteBase", when rules are well-written. At least with sNews.

Well, I tried to be as clear as possible, not for God, but for newbies sake. For ogre sake ;) : Not all sNews install live in the root.  Some of us might have to uncomment that line, change the foldername.
Rewritebase: Without it there seems to be an improvement (on localhost, that is) for this code also:
Quote
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^(.*) $1 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)/$ index.php?category=$1 [L]

Consider:
- The fact that not all server configurations are equal (GoDaddy is a real pain).
- 'Slash handling' might be needed for different reasons and need different solutions. The code in this post might be great for Google, while bad for existing, 'slash-less' but valuable inbound links.
- Typos like this don't occur that often.

I see no all purpose solution here, that works flawless on all servers. I prefer targeting issues as they occur, unless someone has figured out an all purpose solution.
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Armen on August 03, 2008, 05:17:17 am
Well configured server will swallow anything that's supposed to work, according to manuals.

Too bad your server (for a mysterious reason) doesn't process my directives.

BTW, Sven, any problems on your side with the updated htaccess rules?
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Joost on August 03, 2008, 05:29:35 am
Sorry Armen,

But it is not about (any of) my server(s).I don't need a solution. Didn't you read or understand my post?  ???
And there is no such thing as one good configuration. People have all kinds of hosting, depending on that particular server configuration.

Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Sven on August 03, 2008, 10:00:49 am
Hi Dudes,
it works okey for me on 1and1 servers:
- http://www.hiseo.fr/ - http://www.hiseo.fr/index.php
- http://www.hiseo.fr/debuter-avec-un-redacteur-web/ - http://www.hiseo.fr/debuter-avec-un-redacteur-web
it does the job!
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Armen on August 03, 2008, 11:24:03 am
Sorry Armen,

But it is not about (any of) my server(s).I don't need a solution. Didn't you read or understand my post?  ???
And there is no such thing as one good configuration. People have all kinds of hosting, depending on that particular server configuration.



Didn't mean to offend you, Joost.

It's just that I'm now using this solution with 3 different hosting companies and at home. No problems. Different OS'es (Win, FreeBSD, Ubuntu-linux, Bare-Unix - driven servers).

You've done good, Joost. Thank you. And for testing.

Indeed, this solution is not 100% fail-proof because of different server configurations and one day I hope to come up with one really universal solution.
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Sasha on August 04, 2008, 07:11:19 am
what i said before "lets make snews1.6 perfect" also welcome back Armen :) and Joost is a great guy :)
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Sven on August 04, 2008, 01:48:29 pm
Oups !
There's a little problem on keyword search.
The training slash is being applied so search as -http://www.hiseo.fr/tag/wai
gives -http://www.hiseo.fr/tag/wai/
and of course it doesn't work out.
Is there a way to fix that issue in the .htaccess file or in the Mod itself?
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Armen on August 04, 2008, 02:06:42 pm
Oups !
There's a little problem on keyword search.
The training slash is being applied so search as -http://www.hiseo.fr/tag/wai
gives -http://www.hiseo.fr/tag/wai/
and of course it doesn't work out.
Is there a way to fix that issue in the .htaccess file or in the Mod itself?

Just add preg_replace() to a string inside keyword search function to get rid of a trailing slash from the search keyword:

Quote
$var = preg_replace("/\/$/","",$var);

where $var is a variable with XSS cleaned search quiery.

Example:

Quote
<?php
$var = "key/";
$var = preg_replace("/\/$/","",$var);
echo $var;
?>

Will output "key" without a trailing slash.

BTW, it's a good idea to get rid of all slashes from any user-specified quiery. Makes it safer. To clear user's input of all slashes, just remove the "$" (dollar) sign from preg_replace.
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Sven on August 04, 2008, 02:52:34 pm
Okey okey...
I wish good holidays dear ogre. :D
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Sven on August 04, 2008, 03:36:02 pm
didn't find where to paste your piece of code into the function:
Quote
//SEARCH ENGINE ONLY ON KEYWORDS_META
function search_keywords($word) {
echo '<h2>'.l(search_results).'</h2>';
   $word = clean(cleanXSS(dashstrip($word)));
    $query = "SELECT category,seftitle,title,keywords_meta FROM ".db('prefix')."articles WHERE published = 1 AND keywords_meta LIKE '%$word%' ORDER BY date DESC";
    $result = mysql_query($query);
    $numrows = mysql_num_rows($result);
    if (!$numrows) {
      echo '<p>'.l('noresults').' <span class="gras">'.$word.'</span>.</p>';
   }
   else {
            $goodres = 0;   
       while ($r = mysql_fetch_array($result)) {
                $labels = explode(',', $r['keywords_meta']);
                foreach ($labels as $labels){
                    $label = $labels;
                    if($label == $word){
            $date = date(s('date_format'), strtotime($r['date']));
             $finetext = strip_tags($text, '<h1><h2><h3><h4><h5><dl><dt><dd><ul><li><ol><p><hr /><br /><em>');
             $categorySEF = find_cat_sef($r['category']);
           if ( $categorySEF == l('home_sef') )
           {
                      echo '<p><a href="'.db('website').$r['seftitle'].'/">'.$r['title'].'</a></p>';
           }
           else
           {
                      echo '<p><a href="'.db('website').$categorySEF.'/'.$r['seftitle'].'/">'.$r['title'].'</a></p>';
           }
                           $goodres = $goodres + 1;   
                    }
                }
            }
            echo '<p>( '.$goodres.' '.l('resultsfound').'  <span class="gras">'.$word.'</span>)</p>';
   }
   echo '<p><br /><a href="'.db('website').'">'.l('backhome').'</a></p>';
   $searched = true;
}
???
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Sven on August 05, 2008, 09:34:38 am
the ogre is gone on vacations. 8)
Anyone can tell me where this
Code: [Select]
$var = preg_replace("/\/$/","",$var); must be placed?
This a real pain in the a**: my visitors do use the tags cloud machinery. :-\
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Armen on August 05, 2008, 08:14:18 pm
Find

Code: [Select]
$word = clean(cleanXSS(dashstrip($word)));
Right after it add:

Code: [Select]
// To get rid of the last trailing slash
$word = preg_replace("/\/$/","",$word);

BTW, I found a hot spot (Wi-Fi) in Ukraine =)
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Sven on August 06, 2008, 08:14:52 am
Okey Mr Ogre
it works really fine
Thanks a lot.
BTW, I found a hot spot (Wi-Fi) in Ukraine =)
Do you mind to receive a karma in Ukraine? ;D
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Rui Mendes on August 08, 2008, 02:00:45 am
Hello my friends.

I tried your code in localhost, but not works.  :(

I fix your code.
Quote
<IfModule mod_php4.c>
    php_value session.use_trans_sid 0
</IfModule>
<IfModule mod_security.c>
SecFilterEngine Off
SecFilterScanPOST Off
</IfModule>
RewriteEngine On
#RewriteBase /sNews16
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^(.+) $1 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.+)/$ index.php?category=$1 [L]
RewriteCond %{REQUEST_URI} !/$
RewriteRule (.*) %{REQUEST_URI}/ [R=301,L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.+)/$ $1/ [R=301,L]

Now this code works fine with me. Can you try ?
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Joost on August 08, 2008, 02:07:03 am
Rui, which code did you try?
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Rui Mendes on August 08, 2008, 02:20:41 am
Hello my friend, how are you?

I tried all code (from you, Armed, Fun, Sven) from this post, but not redirect well on localhost (with xampp in windows).

I use your code, just change your first quote
Quote
RewriteRule ^(.+)/$ $1/ [R=301,L]

and add this 2 lines too
Quote
RewriteCond %{REQUEST_URI} !/$
RewriteRule (.*) %{REQUEST_URI}/ [R=301,L]
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Joost on August 08, 2008, 02:36:50 am
Hello Rui,

Works nice, except for one flaw:
- sNews in directory, for instance localhost/snews16/
- Nonexisting localhost/snews1 leads to 500 (internal server error).


Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: johnflower on February 14, 2011, 09:42:38 pm
thanks a mil for this! searched for very long for a redirect for index.php, and got the trailing slash fix to boot!

Appreciated!
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: johnflower on February 14, 2011, 09:52:51 pm
oops, i spoke too soon.

um, i can't access some of the pages and categories on my site. can someone perhaps help out on this? i have the following in my htaccess:

Code: [Select]
<IfModule mod_php4.c>
    php_value session.use_trans_sid 0
</IfModule>
<IfModule mod_security.c>
SecFilterEngine Off
SecFilterScanPOST Off
</IfModule>
RewriteEngine On
#RewriteBase /
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index\.php\ HTTP/
RewriteRule ^index\.php$ http://%{HTTP_HOST}/ [R=301,L]
RewriteCond %{REQUEST_URI} ^/[^\.]+[^/]$
RewriteRule ^(.*)$ http://%{HTTP_HOST}/$1/ [R=301,L]
RewriteRule ^([a-z_]+)/([^/]+) index.php?category=$1&title=$2 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([a-z_]+)/([^/]+)/([^/]+)/ index.php?category=$1&title=$2&commentspage=$3 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([a-z0-9_-]+)/([0-9]+)/ index.php?category=$1&nbsp;articlespage=$2 [L]
RewriteCond %{HTTP_HOST} ^papertrading\.co.za [nc]
RewriteRule (.*) http://www.papertrading.co.za/$1 [r=301,nc,L]

Redirect permanent /rss-articles http://feeds.feedburner.com/PaperTrading
Redirect permanent /rss-comments http://feeds.feedburner.com/PaperTrading
Redirect permanent /rss-pages http://feeds.feedburner.com/PaperTrading

site, as can be seen in the code, is www.papertrading.co.za. clicking on archive, contact or any category takes me to an error page.

UPDATE: it seems to be breaking my site completely. the layout is falling apart and images aren't displaying correctly, or not at all.
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: Keyrocks on February 15, 2011, 02:36:14 am
Hmmm... best advice I can suggest is to restore your .htaccess file back to what it was before you modified it.
What you have there is causing the problem.

If you are working with a 1.6 single user package, the first four Condition-Rule string-sets after the #RewriteBase / string should be as follows (these are the latest ones I have kicking around), with absolutely no changes:
Code: [Select]
#RewriteBase /
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^(.*) $1 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php?category=$1 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([a-z_]+)/([^/]+) index.php?category=$1&title=$2 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([a-z_]+)/([^/]+)/([^/]+)/ index.php?category=$1&title=$2&commentspage=$3 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([a-z0-9_-]+)/([0-9]+)/ index.php?category=$1&nbsp;articlespage=$2 [L]

If you wish to add any Condition-Rule stirng-sets of your own, they must come after the above, and not alter or conflict with what the above are meant to accomplish.
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: johnflower on February 15, 2011, 07:05:14 am
Hi Keyrocks,

Thanks for the reply. OK, will revert back to the original. May even perhaps check out Snews 1.7, although I'm reluctant.

Have a good day!
Title: Re: [MOD] Avoiding duplicate content (trailing slash issue)
Post by: johnflower on February 16, 2011, 03:47:18 pm
Ok I took the opportunity to switch over to 1.7. i must admit, i hated the first 1.7, but it seems the current release is quite nice. resembles 1.6 in many ways.

Thanks again for the help!