automated transfers ?

I wish to do automated transfers of web content files from my computer to dreamhost via rsync, using public/private keys without pass-phrases. The transfers are initialized at random intervals so I will not be using cron for this process. Also I wish to run a script that processs a SQL query every time this transfer takes place. Is this possible? If not, what is the best way to do automated data uploads?

Uh, you can do it. A number of people here do. I don't happen to but didn't want to have you waiting around for "the short answer" on a holiday weekend... :)

Ok. Well, that's fine. I spend a lot of time on the computer, so short answer or long answer whatever you got will be fine. So i guess the question is if YOU have any time to share your information that would be appreciated.

Ok your posting are confusing Im not sure what message you trying to getting across.... but i am just mr simple joe newbie so cut me some slack.
I have a pc randomly recording audio clips from a audiosource. These files need to be uploaded to dreamhost. I prefer to use a encrypted transfer, but i will not be able to type a password every time a file transfers. I know rsync has some post transfer execute options, however i don't think they are avail here? When the clips are done recording it needs to be uploaded to a certain directory on dreamhost. and a script needs to be excutaed post transfer on dreamhost that analyzes the filename and directory, disseminates the information, and enters data into MySQL, Then my php webpage will be happy

So anyone wanna throw me some ideas?
thanks

In reply to:
Ok your posting are confusing Im not sure what message you trying to getting across.... but i am just mr simple joe newbie so cut me some slack.

Are you proficient in a shell scripting language?

Are both systems *nix boxes?

Since you are desirous of encrypting the transfer, are you also planning to encrypt the files stored on the DreamHost server?

When you "analyze the filename(s) and director(y/ies)", what "information" are you planning to discern, and how/where is that information to be "desseminated"?

What data is to be entered into MySQL, and what query operations do you envision?

What is the ultimate output for your "php webpage" intended to contain?

You have described a fairly complicated project, and described it in a way that hints at what you really want to do without actually saying it. If you really want to interest others in helping you design it, you need to cut us "some slack" too. wink

Short answer: I'm sure it can be done, but I don't know your skillset.

--rlparker

Are you proficient in a shell scripting language?
Somwhat I learn very fast

Are both systems *nix boxes?
well the only system at home is gentoo, dreamhost is linux

Since you are desirous of encrypting the transfer, are you also planning to encrypt the files stored on the DreamHost server?
No I just want the file transfers to be secure I don't want plain text passwords over the net work of course , I imgine this will require key files or something?

When you "analyze the filename(s) and director(y/ies)", what "information" are you planning to discern, and how/where is that information to be "desseminated"?
ex file is /usa/michigan/stationname/timstampedaudiofile.mp3 I plan on a script the runs after the file transfers into dreamhst to put that data into sql... Unless dreamhost supports remote sql connection..... Dont think soo....

What data is to be entered into MySQL, and what query operations do you envision? As above the only sql command after the transfer would be to put INSERT $country $state etc...
What is the ultimate output for your "php webpage" intended to contain?
links to the audio files with the different meta data values by the link. Im sure i can setup the php fine all it will be is a few SELECT commands.

You have described a fairly complicated project, and described it in a way that hints at what you really want to do without actually saying it. If you really want to interest others in helping you design it, you need to cut us "some slack" too.

Short answer: I'm sure it can be done, but I don't know your skillset.

--rlparker

the part im really wonder about is the dreamhost side accepting automated data transfer and the automated scriptrunning

sorry bout the typos im on my phone

and thanks a lot u are awesome :)

In reply to:
the only system at home is gentoo, dreamhost is linux

Yep, DH is running Debian, so at least you will have the ability to have the same tools (if not exactly the same versions) available.

In reply to:
I just want the file transfers to be secure I don't want plain text passwords over the net work of course , I imgine this will require key files or something?

ssh, scp, or sftp should give you that level of security. If you are only worried about "man in the middle" versus actually protecting the *content*.

In reply to:
ex file is /usa/michigan/stationname/timstampedaudiofile.mp3 I plan on a script the runs after the file transfers into dreamhst to put that data into sql... Unless dreamhost supports remote sql connection..... Dont think soo....

Of course you will have the challenge of determining when new file(s) have arrived in order to do the updates only as needed, but I'm thinking you can include that logic as part of the upload script, as DreamHost *does* allow remote MySQL connection (you can edit the MySQL user in the Control Panel -> Goodies -> Manage MySQL screen to specify remote hosts that my be used by that user to connect to a given database).

In reply to:
the part im really wonder about is the dreamhost side accepting automated data transfer and the automated scriptrunning

Both these issues should be manageable depending upon how you approach implementation.

How would DreamHost even know, and why would they care, whether the data transfer is scripted from your end or whether it is initiated by a user sitting at a keyboard?

I do feel it is worth mentioning that DreamHost, by the nature of it being a shared server hosting platform, is not well suited for running "extensive" automated scripting processes that are not directly related to the serving of pages. Their policies properly disallow the running of persistent processes, and their "procwatch" daemon is ever vigilant in tracking down and killing stuff that gobbles too much CPU/memory/etc.

While you can use "non-abusive" cron jobs for lightweight automation tasks, you can't run anything that is going to listen on a port (a server, etc.), but I don't see anything in what you have described that should required that. Really, I think you can script almost all of it on your own box and just automate the file transfer and MySQL updating.

I'd recommend doing all the scripting you can on your gentoo box (where you have greater control, can run persistent processes at will, etc.) rather than on DreamHost. If you just script your box to connect via ssh/sftp/scp/etc (your choice; scp seems simpler to me, but YMMV) you are halfway there.

To get the rest of the way there, just script your MySQL updating to use a remote MySQL connection and it seems to me you are home free. Of course, I know it's not that simple but that basic architecture seems to meet your needs and won't require any significant or extraordinary scripting at all on the DreamHost side. wink

Anyway, those are my thoughts on the subject. If any of that is helpful or useful at all then great! If not, well, you know the old saying about "free" advice being "worth exactly what you pay for it". wink

Good luck with your project. I think you will probably have little trouble once you get it sorted in your mind what to do on which box.

--rlparker