been messing with this over the psat few days, very cool! made some silly mistakes on the way, thought i'd write them down here so others don't suffer as i have.
-don't start serviio from the console, esp. if you're logged in as admin or root. it messes with the permissions of the log and database. if this happens to you, chmod 777 the log and library folders, and the db folder therein, and delete the db.lck files in the db folder. restart the diskstation, should be happy again.
-chmod 777 /volume1/@tmp and /volume1/@tmp/Serviio, otherwise the transcoder won't work
-most importantly, i had moved all my videos and music to the diskstation via rsync. by default the files were owned and readable only by the admin user, so a quick chmod -R 777 /volume1/video fixed this. yes, videos shouldn't be executable, but i couldn't think of a way to have it seperately chmod subfolders to 777 and files to 644.
hope that helps, and i hope someone can correct my brute force permissions approach.