Block Search Engines

Questions about the Synology Photo Station can be placed here.
Forum rules
1) This is a user forum for Synology users to share experience/help out each other: if you need direct assistance from the Synology technical support team, please use the following form:
https://myds.synology.com/support/suppo ... p?lang=enu
2) To avoid putting users' DiskStation at risk, please don't paste links to any patches provided by our Support team as we will systematically remove them. Our Support team will provide the correct patch for your DiskStation model.
deltaf508
Trainee
Trainee
Posts: 10
Joined: Thu Jul 14, 2011 11:29 pm

Re: Block Search Engines

Postby deltaf508 » Sat Apr 07, 2012 7:25 pm

1. There were two suggestions for the contents of robots.txt. Can anybody provide a bit more information on which one should be used, and why?

2. The advice was to place the robots.txt file into the /web directory. Since this is not a normal share, what do you have to do to get that ...telnet in? Any simple instructions here would be very helpful. (Not looking for a Linux tutorial, just the outline of what to do/where to go.


Where do you put the robots.txt ?
I've used it before and put it in the folder where index.html or index.php is. But there's no html in the /volume1/photo folder ...
And /volume1/web is used by Web Station, not Photo Station as far as I can see.


To all Synology the guru's out there - Sorry to keep bringing up this issue, but I'd like to hear some more responses on these questions specifically...I'm wondering the same things here. I have designed web pages, etc, but I don't see the folders you are talking about. I'm fine telneting or whatever, but just want to make sure I'm doing this right! I'm not comfortable opening up my ports to my photo station (password protected or not) until this is done.

Also, is there a way to modify the main template/photo page without using the customized "settings page". I realize this would be more advanced but am just curious.

Thanks in advance everyone! The more I use my synology the more I love it!
User avatar
Goner
Seasoned
Seasoned
Posts: 590
Joined: Tue Mar 06, 2012 2:27 pm
Location: Rotterdam, Netherlands

Re: Block Search Engines

Postby Goner » Sat Apr 07, 2012 8:20 pm

Goner wrote:But there's no html in the /volume1/photo folder ...

it could be this directory : /volume1/@appstore/PhotoStation/photo
there's an index.php in there. But I can't find it in the Apache .conf files

NAS : DS212j with 2 ST2000DL003 in SHR / DSM 5.2-5644 Update 5
LAN : Fritz!Box 7170, 5 Devolo 200/500Mbps homeplugs, 2 5-port switches, Maxxter ACT-WNP-RP-002
HW : Raspberry Pi 2B & 3B, Conceptronic CHD3NET, ACRyan Playon!HD, Eminent EM7075dts, Wii, Wii U, PS2, D-Link DCS-930L
OS : Linux Mint 16 Cinnamon
deltaf508
Trainee
Trainee
Posts: 10
Joined: Thu Jul 14, 2011 11:29 pm

Re: Block Search Engines

Postby deltaf508 » Sun Apr 15, 2012 5:55 am

Ok, so to follow up on this. I'm still not "exactly" sure what to do here. Now that I know where to go (I think) someone correct me if I'm wrong: "/volume1/@appstore/PhotoStation/photo/" I'm trying to create the robots.txt file by using the vi editor.

*vi robots.txt
*add stuff to txt file
*save via command ":wq"

I get the error message "robots.txt" permission denied"

It can't be that hard here! So can someone explain what else I need to do? Also, what about all the individual user photostations (ie. photostation/~username/photo) If I share one of these out over the internet...where does the robots.txt file go for these?

Also, in regards to the html code in the index.php file. I'd still like to know the use cases for that also.

Thanks!
Last edited by deltaf508 on Sun Apr 15, 2012 6:30 pm, edited 1 time in total.
henkg
Seasoned
Seasoned
Posts: 566
Joined: Sat Sep 25, 2010 8:16 am
Location: Netherlands

Re: Block Search Engines

Postby henkg » Sun Apr 15, 2012 7:35 am

deltaf508 wrote:"/volume1/@appstore/PhotoStation/photo/" I'm trying to create the robots.txt file by using the vi editor.
I get the error message "robots.txt" permission denied"

Same here. Logged in as root I can cd to /volume1/@appstore. The command ls shows -among others- the Photostation directory.
Within the directory @appstore, cd Photostation says: 'cd: can't cd to Photostation'.
Any ideas?
DS412+. 4x WD40EFRX, RAID5. DSM 6.0-8451 u9
DS210j. 2x HD204UI, RAID1. DSM 5.2-5644 u2
Windows 10. Router EPC3940. Switch TP-Link TL-SG1008D. Mede8er MED600X3D. UPS APC BE700G-GR.
User avatar
HarryPotter
Honorary Moderator
Honorary Moderator
Posts: 18868
Joined: Mon Oct 23, 2006 12:48 pm
Location: Switzerland

Re: Block Search Engines

Postby HarryPotter » Sun Apr 15, 2012 11:42 am

PhotoStation is not equal to Photostation...... it is case-sensitive :!:
*Please do not Private Message me for support questions; leave it on the forum so all members can learn. Thanks!*

DS718+ / DSM 6.1.3-15152-7 / ST4000VN000-2AH166 / 16 GB RAM
DS713+ / DSM 6.1.3-15152-7 / HD501LJ + HD502IJ (RAID0)
DS415+ / DSM 6.1.3-15152-7

2 Squeezebox 3 + Boom

APC Smart UPS SUA750i
User avatar
Goner
Seasoned
Seasoned
Posts: 590
Joined: Tue Mar 06, 2012 2:27 pm
Location: Rotterdam, Netherlands

Re: Block Search Engines

Postby Goner » Sun Apr 15, 2012 1:30 pm

deltaf508 wrote:*vi robots.txt
*add stuff to txt file
*save via command "!wq"

after adding the text to the file, use Esc to exit edit mode and then type :wq followed by Enter

NAS : DS212j with 2 ST2000DL003 in SHR / DSM 5.2-5644 Update 5
LAN : Fritz!Box 7170, 5 Devolo 200/500Mbps homeplugs, 2 5-port switches, Maxxter ACT-WNP-RP-002
HW : Raspberry Pi 2B & 3B, Conceptronic CHD3NET, ACRyan Playon!HD, Eminent EM7075dts, Wii, Wii U, PS2, D-Link DCS-930L
OS : Linux Mint 16 Cinnamon
User avatar
myCloud
Skilled
Skilled
Posts: 648
Joined: Fri Mar 23, 2012 11:28 am

Re: Block Search Engines

Postby myCloud » Sun Apr 15, 2012 4:57 pm

If you want some visibility and control over what Google does with your site, go to their Webmaster Tools and set it up for your site. It will generate a key file you have to put in the root of your website to prove that you have rights to administer it. This helps a lot.

Bing, which now also manages Yahoo! has a similar thing.
DS 1512+ w/3GB, 5 x 3TB Seagate ST3000DM001 8.2TB RAID 6, half files/half Time Machine.
Icy Dock MB559U3S-1SB enclosure w/4TB Hitachi UltraStar via USB 3 for files backup
UVERSE to AirPort Extreme + 2 AirPort Express w/speakers. TRENDnet TV-IP312WN camera
CyberPower CP1500PFCLCD Sine Wave UPS
DSM 4.1-2661 w/SSH + SFTP, VPN Server, Syslog Server, Media Server, Mail Server, Mail Station,
Audio Station, Surveillance Station, Photo Station, Web Station - DS Apps on iPad & iPod Touch.
deltaf508
Trainee
Trainee
Posts: 10
Joined: Thu Jul 14, 2011 11:29 pm

Re: Block Search Engines

Postby deltaf508 » Sun Apr 15, 2012 6:27 pm

after adding the text to the file, use Esc to exit edit mode and then type :wq followed by Enter


I did this....Any other ideas I can try?

If you want some visibility and control over what Google does with your site, go to their Webmaster Tools and set it up for your site. It will generate a key file you have to put in the root of your website to prove that you have rights to administer it. This helps a lot.


Thanks for the info, I may check it out, but I don't think this would effect specifically editing this file on our synology (please correct me if I'm wrong here)?

Thanks!
User avatar
myCloud
Skilled
Skilled
Posts: 648
Joined: Fri Mar 23, 2012 11:28 am

Re: Block Search Engines

Postby myCloud » Sun Apr 15, 2012 6:35 pm

IIRC, part of it checks the validity of your robots.txt file against what Google expects to see.
DS 1512+ w/3GB, 5 x 3TB Seagate ST3000DM001 8.2TB RAID 6, half files/half Time Machine.
Icy Dock MB559U3S-1SB enclosure w/4TB Hitachi UltraStar via USB 3 for files backup
UVERSE to AirPort Extreme + 2 AirPort Express w/speakers. TRENDnet TV-IP312WN camera
CyberPower CP1500PFCLCD Sine Wave UPS
DSM 4.1-2661 w/SSH + SFTP, VPN Server, Syslog Server, Media Server, Mail Server, Mail Station,
Audio Station, Surveillance Station, Photo Station, Web Station - DS Apps on iPad & iPod Touch.
User avatar
Goner
Seasoned
Seasoned
Posts: 590
Joined: Tue Mar 06, 2012 2:27 pm
Location: Rotterdam, Netherlands

Re: Block Search Engines

Postby Goner » Sun Apr 15, 2012 10:53 pm

myCloud wrote:IIRC, part of it checks the validity of your robots.txt file against what Google expects to see.

Yes, but the problem is that it looks like he cannot create a robots.txt ... :?

deltaf508 wrote:I did this....Any other ideas I can try?

And still no robots.txt in your photo folder ?

NAS : DS212j with 2 ST2000DL003 in SHR / DSM 5.2-5644 Update 5
LAN : Fritz!Box 7170, 5 Devolo 200/500Mbps homeplugs, 2 5-port switches, Maxxter ACT-WNP-RP-002
HW : Raspberry Pi 2B & 3B, Conceptronic CHD3NET, ACRyan Playon!HD, Eminent EM7075dts, Wii, Wii U, PS2, D-Link DCS-930L
OS : Linux Mint 16 Cinnamon
deltaf508
Trainee
Trainee
Posts: 10
Joined: Thu Jul 14, 2011 11:29 pm

Re: Block Search Engines

Postby deltaf508 » Sun Apr 15, 2012 11:17 pm

Figured it out! I was telnetting in as admin assuming that would the the highest authoritative account on my synology. Evidentially not. I read root somewhere and it hit me. Telnet in with "root" using the admin password and bingo. It created it like it should. So since there does not seem to be a definitive explanation of what to create and where I'll post what I've done and everyone can let me know if this is right or not. Seems there are still others wondering how to do this as well.

  • Enable telnet on NAS
  • Telnet in as root using admin password
  • Use command "cd /volume1/@appstore/PhotoStation/photo" to go to appropriate folder
  • Create robots.txt file using vi edtitor as follows:
    "vi robots.txt" at command prompt
    (enter i) to go into (i)nput mode then type:
    User-agent: *
    Disallow: /
    press (esc) to exit out of (i)nput mode
    ":wq" to save the file
  • enter "ls -la" at command prompt to confirm your robots.txt file is there.

Is this correct? If so, it seems synology could put a simple check box in the gui somewhere and automatically create this file?

Now onto my next question which was does this prevent search engines from reporting just your "main" photo station or does it also prevent the user account photo stations (ie. photostation/~username/photo)? (which is really what I'm after since I will be sharing one of those out to family.)

Thanks everyone!
benjibarnicals
Enlightened
Enlightened
Posts: 456
Joined: Sun Mar 18, 2007 10:49 pm

Block /webman/ DSM Desktop from Search Engines

Postby benjibarnicals » Sun Apr 15, 2012 11:17 pm

Hi all,

I've started to see my domain to my Nas appearing in the Google index. So far it's just the /webman/ and domain root showing, although these pages are running under HTTPS and they are password protected i'd prefer if Google didn't try to index the Nas, these pages or anything underneath it.

Is there a simple way in DSM to stop this?



Thanks,

Ben
deltaf508
Trainee
Trainee
Posts: 10
Joined: Thu Jul 14, 2011 11:29 pm

Re: Block Search Engines

Postby deltaf508 » Sun Apr 15, 2012 11:21 pm

Also, what are the advantages of using the html code instead, and where would you implement this if used?

<HTML>
<HEAD>
<meta name="robots" content="noindex,nofollow">
<TITLE></TITLE>
</HEAD>
<BODY>
</BODY>
</HTML>


Thanks again.
henkg
Seasoned
Seasoned
Posts: 566
Joined: Sat Sep 25, 2010 8:16 am
Location: Netherlands

Re: Block Search Engines

Postby henkg » Mon Apr 16, 2012 5:58 am

HarryPotter wrote:PhotoStation is not equal to Photostation...... it is case-sensitive :!:

I know Linux is case sensitive, but I didn't notice the capital S in PhotoStation, and looked over it again and again. I feel stupid.:oops:
Thanks Harry.
DS412+. 4x WD40EFRX, RAID5. DSM 6.0-8451 u9
DS210j. 2x HD204UI, RAID1. DSM 5.2-5644 u2
Windows 10. Router EPC3940. Switch TP-Link TL-SG1008D. Mede8er MED600X3D. UPS APC BE700G-GR.
gwp
I'm New!
I'm New!
Posts: 3
Joined: Sun Mar 04, 2012 8:10 pm

Re: Block Search Engines

Postby gwp » Sat Apr 21, 2012 6:04 am

deltaf508 wrote:Figured it out! I was telnetting in as admin assuming that would the the highest authoritative account on my synology. Evidentially not. I read root somewhere and it hit me. Telnet in with "root" using the admin password and bingo. It created it like it should. So since there does not seem to be a definitive explanation of what to create and where I'll post what I've done and everyone can let me know if this is right or not. Seems there are still others wondering how to do this as well.

  • Enable telnet on NAS
  • Telnet in as root using admin password
  • Use command "cd /volume1/@appstore/PhotoStation/photo" to go to appropriate folder
  • Create robots.txt file using vi edtitor as follows:
    "vi robots.txt" at command prompt
    (enter i) to go into (i)nput mode then type:
    User-agent: *
    Disallow: /
    press (esc) to exit out of (i)nput mode
    ":wq" to save the file
  • enter "ls -la" at command prompt to confirm your robots.txt file is there.

Is this correct? If so, it seems synology could put a simple check box in the gui somewhere and automatically create this file?

Now onto my next question which was does this prevent search engines from reporting just your "main" photo station or does it also prevent the user account photo stations (ie. photostation/~username/photo)? (which is really what I'm after since I will be sharing one of those out to family.)

Thanks everyone!


I am trying to get my photostation access set up for HTTPS. I was able to successfully create the robots.txt file in /volume1/@appstore/PhotoStation/photo. I then went to http://tool.motoricerca.info/robots-checker.phtml to test it, but it can only find it if I have port 80 forwarded on my router and enabled on my DS. It will not find it if I only have 443 enabled and forwarded for HTTPS. Am I ok anyway? I don't want google indexing my pictures (all will be behind usernames/passwords). I know just enough to be dangerous, but I want to ensure that my DS is functioning the way I want it to (secure connections/ports only). Does what I am asking make sense?

Return to “Photo Station”

Who is online

Users browsing this forum: No registered users and 2 guests