Downloading stats with Summary - Page 2

percepts · 27th May 2008, 01:13 PM

And one last thing. If summary uses http by default and you cannot make it login using ftp, then it won't work. You must login via ftp to access /var/log/httpd/ directly

If that is the case, then you may have to set up a cron job to copy your logs to an http accessible area of your domain and then use summary to do what it does.

comedy79 · 27th May 2008, 08:37 PM

Quote:

Originally Posted by percepts

And one last thing. If summary uses http by default and you cannot make it login using ftp, then it won't work. You must login via ftp to access /var/log/httpd/ directly

If that is the case, then you may have to set up a cron job to copy your logs to an http accessible area of your domain and then use summary to do what it does.

Yes, I can log in via ftp (the different failure message is shown in a later post). I was just trying http after multiple failures with ftp.

comedy79 · 27th May 2008, 08:44 PM

Quote:

Originally Posted by desquinn

if you login by ftp to homedir and the do a change directory to /var/log/httpd then that will work. However your path is resulting in a change directory command to var/log/httpd which equates to /home/xx_siteuser/mainwebsite_html/var/log/httpd/

Further to my confusion about the true path to the log directory, the attachment shows the path to the logs when I log in with Fetch, a Mac ftp client.

So I am having a lot of trouble working out what my path should be using your instructions, sorry!

MrBen · 27th May 2008, 09:28 PM

You need to ignore desquinn's path for the logs. They're wrong. As you have seen in Fetch and from some of the other posts in this thread /var/log/httpd is the correct path.

Ben

percepts · 27th May 2008, 10:20 PM

Quote:

Originally Posted by comedy79

Yes, I can log in via ftp (the different failure message is shown in a later post). I was just trying http after multiple failures with ftp.

Yes but it says :

'ftp://mydomain.com/var/log/httpd/'

whereas it should be looking at:

'ftp://ftp.mydomain.com/var/log/httpd/'

MrBen · 27th May 2008, 10:30 PM

If the DNS for the domain is at UH there is no difference between mydomain.com and ftp.mydomain.com. You could just use the server IP address or even another site's domain name and it would still work. The login is the identifier.

Ben

percepts · 27th May 2008, 11:31 PM

Quote:

Originally Posted by MrBen

If the DNS for the domain is at UH there is no difference between mydomain.com and ftp.mydomain.com. You could just use the server IP address or even another site's domain name and it would still work. The login is the identifier.

Ben

are you sure about that. I know it works for your http server but mydomain.com will point at the http server and not the ftp server as far as I'm aware.

MrBen · 27th May 2008, 11:54 PM

I am

So long as the host record points at the correct UH server it makes diddly squat difference for any of the services except HTTP and HTTPS. All the logins are user@domain.com or user#domain.com so you always know which site someone is related to.

The IP address can also be either a dedicated or shared IP, it makes no difference so long as it is the same UH server. A site with a dedicated IP could still use the shared IP for non-HTTP[S] services. When a site has a dedicated IP address you could point whatever domain you liked at that IP and the same site would be returned (unless you did some .htaccess trickery). This is because the IP address is the identifier in this instance.

The host name used only becomes key for HTTP sites as Apache will match on the host record and then determine which site to load (i.e. the domain name is the identifier). It's obviously the same for HTTPS, but then you also have to consider the host name the SSL certificate was issued to.

This is how users can set-up, upload, and test their sites prior to moving DNS.

The use of mail.domain.com, ftp.domain.com, smtp.domain.com etc in this kind of setup is more about just following convention. It's also less work if you were to say shift email off at some point to someone else but keep the website with UH. You could then just update the appropriate email host records (mail.domain.com and smtp.domain.com) and be away without having to update each individual user's settings (in theory anyway!).

Ben

comedy79 · 28th May 2008, 09:42 AM

Quote:

Originally Posted by MrBen

You need to ignore desquinn's path for the logs. They're wrong. As you have seen in Fetch and from some of the other posts in this thread /var/log/httpd is the correct path.

Ben

That's nice to know, because it's the path I've been using all along. But I still can't get in.

The login I'm using is the one I was given by UH for FTP, one which works perfectly in Fetch for uploading and downloading files from the site.

So I'm still puzzled as to why I can't retrieve my logs.

I'll restart the Mac this evening, just in case this is the cause of my problems. Always a good move, ay?

Ann

desquinn · 28th May 2008, 10:43 AM

Quote:

Originally Posted by comedy79

That's nice to know, because it's the path I've been using all along. But I still can't get in.

The login I'm using is the one I was given by UH for FTP, one which works perfectly in Fetch for uploading and downloading files from the site.

So I'm still puzzled as to why I can't retrieve my logs.

I'll restart the Mac this evening, just in case this is the cause of my problems. Always a good move, ay?

Ann

now I am confused

as well as I thought my paths were correct as I was stating /var/log/httpd/ but also explaining the relationship with the relative directory path to the home directory. xx_siteuser in my paths above relate to the root directory for a site if that is the confusion. Or if it was the relative ftp URL then ok that was a throwaway just to try.

I think your issue is as others and myself are suggesting that you need to login using ftp with this programme and authenticate as the site so that you can pull down the logs.

moving the logs down locally with a scheduled ftp job is another option as is moving them to a web accessible area and then using the program.

You have not answered percepts query about summary apparently requiring gz files straight away. On UH the first and second files are not compressed so if it is looking for these then it may be encountering the wrong file.

i.e. this is a typical listing

Code:

access_log
access_log.1
access_log.2.gz
access_log.3.gz
access_log.4.gz
access_log.5.gz

comedy79 · 28th May 2008, 09:29 PM

Quote:

Originally Posted by desquinn

I think your issue is as others and myself are suggesting that you need to login using ftp with this programme and authenticate as the site so that you can pull down the logs.

moving the logs down locally with a scheduled ftp job is another option as is moving them to a web accessible area and then using the program.

Apart from one attempt via http during my original testing, I have been using ftp all along. I log in with the site login as advised by UH when I joined.

Quote:

Originally Posted by desquinn

You have not answered percepts query about summary apparently requiring gz files straight away. On UH the first and second files are not compressed so if it is looking for these then it may be encountering the wrong file.

I've been using Summary for over four years on lots of different log types. If you look at the error I quoted (reproduced below) you can see that Summary is checking for several different types of header and not encountering any of them:

FTP expected 250 got: '550 var/log/httpd: No such file or directory'
Remote directory not found!
Download failed: Remote directory not found! On 'ftp://dressageit.com/var/log/httpd/'.
Check log header: _.gz
Check log header: A
Check log header: httpd
Read 0 entries in 0:00!
Saving binary report database.
Processing is complete!

Note also that I am using ftp!

It says it is not finding the remote directory, yet I can verify that the exact URI I have entered is (copied and pasted from the Summary configuration page):

ftp://dressageit.com/var/log/httpd/

Ann

comedy79 · 28th May 2008, 09:32 PM

Quote:

Originally Posted by percepts

try

ftp://ftp.mydomain.com/var/log/httpd/

I tried that early in the piece, but will try it again...

FTP expected 250 got: '550 var/log/httpd: No such file or directory'
Remote directory not found!
Download failed: Remote directory not found! On 'ftp://ftp.dressageit.com/var/log/httpd/'.
Check log header: _.gz
Check log header: A
Check log header: httpd
Read 0 entries in 0:00!
Saving binary report database.
Processing is complete!

Nope, no difference.

Euge · 28th May 2008, 10:13 PM

Maybe the user you're logging in with doesn't have access to that directory? Are you logging in with an admin user?

desquinn · 28th May 2008, 10:39 PM

Quote:

Originally Posted by comedy79

Apart from one attempt via http during my original testing, I have been using ftp all along. I log in with the site login as advised by UH when I joined.

I've been using Summary for over four years on lots of different log types. If you look at the error I quoted (reproduced below) you can see that Summary is checking for several different types of header and not encountering any of them:

FTP expected 250 got: '550 var/log/httpd: No such file or directory'
Remote directory not found!
Download failed: Remote directory not found! On 'ftp://dressageit.com/var/log/httpd/'.
Check log header: _.gz
Check log header: A
Check log header: httpd
Read 0 entries in 0:00!
Saving binary report database.
Processing is complete!

Note also that I am using ftp!

It says it is not finding the remote directory, yet I can verify that the exact URI I have entered is (copied and pasted from the Summary configuration page):

ftp://dressageit.com/var/log/httpd/

Ann

Hi Sorry, I was trying to emphasise the authentication not the protocol. You are receiving a 550 error message which is normally a permission denied which could relate to incorrect path/file or an incorrect or omitted username / password.

I saw it checking the different types but none of them to me suggested that it was definitely an uncompressed format although that could be the http.

the error still contains var/log/httpd which is a relative rather than an absolute path as per my previous post.

desquinn · 28th May 2008, 11:01 PM

I have just tried with one of my sites the following:

ncftpget -u user@domain.org -p password ftp://goose.xssl.net/../../var/log/httpd/error_log and it worked ok.

ncftpget is a command line ftpclient so this may work for you as well.

The other thing is to go back to whoever makes summary and see if they can handle absolute urls or a change directory request after logging into a server.

percepts · 28th May 2008, 11:36 PM

one last attempt at getting through...

Have you assumed that the format of the log files on UH servers is the same as on your previous host? By that I mean, have you changed the parameters which summary seems to be checking for which it describes as log headers in your error messages, and have you actually looked at the log format on UH servers to see if they are the same as on your previous host.

These things are highly configurable by whoever sets up apache and not all apache servers output the deafult log format and most setups use their own naming convention for log files.

So, since your error messages seem to be telling all of us except you that it can't find
_.gz
A
httpd

then I suggest you change the parameters that are looking for _.gz to look for .1 and see if that works.

comedy79 · 29th May 2008, 08:49 AM

Quote:

Originally Posted by percepts

Yes but it says :

'ftp://mydomain.com/var/log/httpd/'

whereas it should be looking at:

'ftp://ftp.mydomain.com/var/log/httpd/'

I get identical results with and without the extra ftp.

comedy79 · 29th May 2008, 08:51 AM

Quote:

Originally Posted by desquinn

the error still contains var/log/httpd which is a relative rather than an absolute path as per my previous post.

I can assure you I am using an absolute path. Do you have any suggestions as to what I can try to resolve the problem?

desquinn · 29th May 2008, 08:59 AM

I am not saying you are not

I am saying that it appends to the site which then registers it as a relative path. A normal ftp client will do the following:

Quote:

connect to ftp server
authenticate
change to home dir
list files
CD to /var/log/

I would try the url I was suggesting in my post above if summary does not allow for a post login change of directory. e.g.

Quote:

I have just tried with one of my sites the following:

ncftpget -u user@domain.org -p password ftp://goose.xssl.net/../../var/log/httpd/error_log and it worked ok.

ncftpget is a command line ftpclient so this may work for you as well.

The other thing is to go back to whoever makes summary and see if they can handle absolute urls or a change directory request after logging into a server.

obviously replace error with access

comedy79 · 29th May 2008, 09:09 AM

Quote:

Originally Posted by Euge

Maybe the user you're logging in with doesn't have access to that directory? Are you logging in with an admin user?

Thanks for your response. Yes, I am logging in with the admin user for the domain. The exact same login with an FTP client gives me full access to the directories and I can see all the logs sitting there.

I've been using Summary for over four years and have managed to connect to my logs at several el cheapo sites (different file and folder structures) in the past with few problems. I'm pleased to be with UH and have had wonderful support since joining a fortnight ago. It's just finding the key to getting the logs down into my local directory that's causing difficulties. I'm sure it's a simple little thing that's causing the obstruction, but am having difficulty identifying it.