Block Search Engines

Questions about the Synology Photo Station can be placed here.
Forum rules
1) This is a user forum for Synology users to share experience/help out each other: if you need direct assistance from the Synology technical support team, please use the following form:

https://account.synology.com/support/su ... p?lang=enu



2) To avoid putting users' DiskStation at risk, please don't paste links to any patches provided by our Support team as we will systematically remove them. Our Support team will provide the correct patch for your DiskStation model.
mhdebokx
I'm New!
I'm New!
Posts: 5
Joined: Wed Nov 22, 2006 8:19 pm

Block Search Engines

Unread post by mhdebokx » Wed Nov 22, 2006 8:25 pm

At the moment I'm running the PhotoWeb station at a fixed IP adres at the internet. Somehow different search engines, like Google, have already found the DiskStation. This is causing too much trafic, so I like to block some IP adresses, for example 66.249.65.116.
Is there a way to block some IP adressen or domains?

The DS106e is running behing a LinkSys WRT54G router, but there is also no opportunity to block adressen.

I tried to make an .httaccess where I denied IP 66.249.65.116 and put that file in all photo-share folders, but thus didnot help.

Who can help me?

User avatar
ikeke
Seasoned
Seasoned
Posts: 579
Joined: Wed Oct 25, 2006 2:26 pm
Location: Lille - France

Re: Block Search Engines

Unread post by ikeke » Wed Nov 22, 2006 8:38 pm

mhdebokx wrote:At the moment I'm running the PhotoWeb station at a fixed IP adres at the internet. Somehow different search engines, like Google, have already found the DiskStation. This is causing too much trafic, so I like to block some IP adresses, for example 66.249.65.116.
Is there a way to block some IP adressen or domains?

The DS106e is running behing a LinkSys WRT54G router, but there is also no opportunity to block adressen.

I tried to make an .httaccess where I denied IP 66.249.65.116 and put that file in all photo-share folders, but thus didnot help.

Who can help me?
You can try to create a robots.txt file at the root of the web directory (share called "web").

In this robots.txt insert:
User-agent: *
Disallow: /
It will tell to search engine that you disallow all search engine to explore your website root folder and all sub-directories.
Last edited by ikeke on Wed Nov 22, 2006 11:50 pm, edited 1 time in total.
CS-407 | HD Model: 4x Samsung HD501LJ 500GB Raid5
DS-1513+ | HD Model: 2x Seagate LP 1.5 TB + 2x Seagate LP 2 TB
http://www.nas-forum.com - Communauté Française des utilisateurs de NAS Synology

koez
Novice
Novice
Posts: 52
Joined: Wed Nov 15, 2006 8:41 pm
Location: Finland

Re: Block Search Engines

Unread post by koez » Wed Nov 22, 2006 9:12 pm

Edit: this message was removed since it was invalid.
Last edited by koez on Sun Nov 26, 2006 8:12 pm, edited 1 time in total.

User avatar
NetBoot
Ace
Ace
Posts: 725
Joined: Tue Oct 24, 2006 8:20 pm
Location: Northeastern U.S.

Re: Block Search Engines

Unread post by NetBoot » Wed Nov 22, 2006 11:45 pm

ikeke wrote:
mhdebokx wrote:At the moment I'm running the PhotoWeb station at a fixed IP adres at the internet. Somehow different search engines, like Google, have already found the DiskStation. This is causing too much trafic, so I like to block some IP adresses, for example 66.249.65.116.
Is there a way to block some IP adressen or domains?

The DS106e is running behing a LinkSys WRT54G router, but there is also no opportunity to block adressen.

I tried to make an .httaccess where I denied IP 66.249.65.116 and put that file in all photo-share folders, but thus didnot help.

Who can help me?
You can try to create a robots.txt file at the root of the web directory (share called "web").

In this robot.txt insert:
User-agent: *
Disallow: /
It will tell to search engine that you disallow all search engine to explore your website root folder and all sub-directories.
thumbs up!
Product Model: DS-106
Firmware Version: 2.0.3 - 0640

I have my reasons for my insanity....

User avatar
ikeke
Seasoned
Seasoned
Posts: 579
Joined: Wed Oct 25, 2006 2:26 pm
Location: Lille - France

Unread post by ikeke » Wed Nov 22, 2006 11:52 pm

The name of the file is robots.txt not robot.txt...sorry
Previous message has been edited
CS-407 | HD Model: 4x Samsung HD501LJ 500GB Raid5
DS-1513+ | HD Model: 2x Seagate LP 1.5 TB + 2x Seagate LP 2 TB
http://www.nas-forum.com - Communauté Française des utilisateurs de NAS Synology

User avatar
NetBoot
Ace
Ace
Posts: 725
Joined: Tue Oct 24, 2006 8:20 pm
Location: Northeastern U.S.

Unread post by NetBoot » Thu Nov 23, 2006 2:06 am

here is another example:

index.php

Code: Select all

<HTML>
<HEAD>
<meta name="robots" content="noindex,nofollow">
<TITLE></TITLE>
</HEAD>
<BODY>
</BODY>
</HTML>
Net....
Product Model: DS-106
Firmware Version: 2.0.3 - 0640

I have my reasons for my insanity....

mhdebokx
I'm New!
I'm New!
Posts: 5
Joined: Wed Nov 22, 2006 8:19 pm

Tried ROBOTS.TXT

Unread post by mhdebokx » Sun Nov 26, 2006 6:28 pm

I (already) tried to make a file called ROBOTS.TXT. In put it in a \web\ folder and \photo\ folder as well.

I turned off the Synology for a couple days. When I turned it back on, within 5 minutes Google was searching again :evil:

The Linksys WRT54G can only block outgoing internet access and not the incomming.

:arrow: Is there not a way to make a patch to the synology so we can block those silly search engines ?

User avatar
Pearl
Experienced
Experienced
Posts: 116
Joined: Mon Nov 06, 2006 10:21 am
Location: HK 東方之珠 :)

Unread post by Pearl » Fri Dec 08, 2006 7:05 am

I'm interested/concern about this subject also. I run my DS behind the same router (WRT54G). Currently, all my albums are shared privately (i.e. requires login) in my Photostation.

Would this prevent the search engine from indexing my photos?
Synology Model: DS212j (Installed: Oct-2012)
Firmware Version: DSM 4.3-3827 Update 7 (Updated: 12-Sep-2014)
HDD1 Model: Seagate ST2000DM001 2.0TB
HDD2 Model: Seagate ST2000DM001 2.0TB
--------------------------------------------------------
Synology Model: DS-106 (Dead) with an external 120mm fan
Firmware Version: DSM 2.2-0959 (13-Nov-2009)
HDD Model: Samsung HD204UI 2.0TB

itf
Trainee
Trainee
Posts: 15
Joined: Tue Nov 07, 2006 1:32 pm
Location: Manchester
Contact:

robots.txt not ROBOTS.TXT

Unread post by itf » Fri Dec 08, 2006 2:10 pm

Just out of interest did you call the file ROBOTS.TXT or robots.txt, because on Linux this does make a difference.

Linux is normally case sensitive!

Also if you want a web resource that details how to use the robots.txt here is a site I often use: http://www.robotstxt.org/wc/robots.html

koez
Novice
Novice
Posts: 52
Joined: Wed Nov 15, 2006 8:41 pm
Location: Finland

Unread post by koez » Fri Dec 08, 2006 2:23 pm

And if I want to allow access from specific domains only, can it be done via some /etc/hosts.allow file (if it exists on this device)?

--Linux noob

deejaysly
Beginner
Beginner
Posts: 20
Joined: Wed Feb 07, 2007 4:55 pm

Unread post by deejaysly » Wed May 23, 2007 9:54 am

With regards to robots.txt, it does work. Googlebot, for instance, needs to see that the robots.txt file is there and inspect it. By turning off your DS, Googlebot (or any bot) won't know any better because it hasn't had a chance to inspect your robots.txt file. Once it inspects the robots.txt file, it does take some time for that bot to realise that your site is restricted! It's a bit like a daily database update thing!

That's what I have found anyway..... After a few days of restricting access, Googlebot no longer crawls my DS.

[EDIT]: Google says "We check for a new robots.txt file approximately once per day"

Simmayor
Beginner
Beginner
Posts: 25
Joined: Sat Jul 07, 2007 6:58 pm
Location: The Netherlands
Contact:

Unread post by Simmayor » Mon Sep 17, 2007 1:14 pm

How can i implement the Robots.txt?

Just place the txt file in the root of the photo folder on the DS?

Simmayor
DS-207 128MB, 2x Samsung HD501LJ 500GB drives in RAID 1
Firmware version 2.0.3. - 0510
Enabled Services : Photostation, Webserver, MySQL, FTP
--> Heatsink mod on DiskStation systemboard <--

User avatar
Toxic
Knowledgeable
Knowledgeable
Posts: 306
Joined: Wed Jun 06, 2007 6:19 pm
Location: Belfast
Contact:

Unread post by Toxic » Fri Sep 21, 2007 8:25 pm

Regards Simon
http://www.linksysinfo.org
Synology CS407 - DSM 3.0-1354
Cisco ASA5505 v8.3(4)2 - ASDM v6.3(4)53 512Mb/512Mb

JustMy2Cents
Student
Student
Posts: 68
Joined: Mon Sep 15, 2008 8:34 am

Re: Block Search Engines

Unread post by JustMy2Cents » Tue Apr 14, 2009 1:31 pm

I recommend putting the Photostation on another port. I've done so. And it solved everything..

Edit For future reference: I've done this via a port-redirection in the router

LikeAnimal28
I'm New!
I'm New!
Posts: 1
Joined: Wed Sep 16, 2009 4:04 pm

Re: Block Search Engines

Unread post by LikeAnimal28 » Wed Sep 16, 2009 4:07 pm

Newbie here, but can the Search Engines really see Photostation if all your accounts that are setup require password.

Post Reply

Return to “Photo Station”