http headers and sockets

Message

debianSuiz · #1 Post by **debianSuiz** » 2008-05-05 13:46

Hello!! I'm trying to get the binary data of an remote image by socket and http headers and save the binary data in a file. When i print the received data, the server gives me this output:
HTTP/1.1 400 Bad Request
Content-Length: 0
Server: Network Camera with Pan/Tilt
Connection: close

/1.1 400 Bad Request
Content-Length: 0
Server: Network Camera with Pan/Tilt

Here is the source code:

#include <unistd.h>
#include <arpa/inet.h>
#define tam 100
#define tam_recv 300
#define port 80
#define host "192.168.0.100"
//#define host "127.0.0.1"

int main(void)
{
int sd,newsd,error,cantidad;
char buff[tam];
char contenido[]="";
sockaddr_in serveraddr;
sd=socket(AF_INET,SOCK_STREAM,0);

if(sd==-1)
printf("Error al crear socket\n");

serveraddr.sin_family=AF_INET;
serveraddr.sin_addr.s_addr=inet_addr(host);
serveraddr.sin_port=htons(port);
error=connect(sd,(struct sockaddr*)&serveraddr,sizeof(serveraddr));
if(error<0)
{
printf("Error al conectarse\n");
}
else printf("Conexion establecida\n");

sprintf(buff,"GET /cgi-bin/video.jpg HTTP/1.1 200\r\n");
write(sd,buff,sizeof(buff));
/*sprintf(buff,"HOST:192.168.0.3\r\n");
write(sd,buff,sizeof(buff));*/
//sprintf(buff,"text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5\r\n");
sprintf(buff,"Content-type: image/jpg\r\n");
write(sd,buff,sizeof(buff));
sprintf(buff,"Content-length: %d\n\n",50000);
write(sd,buff,sizeof(buff));
//write(sd,contenido,sizeof(contenido));

while(cantidad=read(sd,buff,tam))
{
printf("%s",buff);
}

close(sd);

return 1;
}

Why i don'nt get the binary data??
thank you very much!!

debianSuiz · #2 Post by **debianSuiz** » 2008-05-05 13:58

Searching I found that i have to sent more headers, this is the new source code:

#include <stdio.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <sys/types.h>
#include <unistd.h>
#include <arpa/inet.h>
#define tam 150
#define port 80
#define host "127.0.0.1"

int main(void)
{
int sd,newsd,error,cantidad;
char buff[tam];
char contenido[]="";
sockaddr_in serveraddr;
sd=socket(AF_INET,SOCK_STREAM,0);

if(sd==-1)
printf("Error al crear socket\n");

serveraddr.sin_family=AF_INET;
serveraddr.sin_addr.s_addr=inet_addr(host);
serveraddr.sin_port=htons(port);
error=connect(sd,(struct sockaddr*)&serveraddr,sizeof(serveraddr));
if(error<0)
{
printf("Error al conectarse\n");
}
else printf("Conexion establecida\n");

sprintf(buff,"GET /cgi-bin/video.jpg HTTP/1.1\n");
write(sd,buff,sizeof(buff));
sprintf(buff,"Host: 192.168.0.100\n");
write(sd,buff,sizeof(buff));
sprintf(buff,"User-Agent: Mozilla/5.0 (X11; U; Linux i686; es-ES; rv:1.8.1.13) Gecko/20080325 Ubuntu/7.10 (gutsy) Firefox/2.0.0.13\n");
write(sd,buff,sizeof(buff));
sprintf(buff,"Accept: text/html,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5\n");
write(sd,buff,sizeof(buff));
sprintf(buff,"Accept-Language: es-es,es;q=0.8,en-us;q=0.5,en;q=0.3\n");
write(sd,buff,sizeof(buff));
sprintf(buff,"Accept-Enconding: gzip,deflate\n");
write(sd,buff,sizeof(buff));
sprintf(buff,"Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\n");
write(sd,buff,sizeof(buff));
sprintf(buff,"Keep-Alive: 300\n");
write(sd,buff,sizeof(buff));
sprintf(buff,"Connection: keep-alive\n");
write(sd,buff,sizeof(buff));

write(sd,contenido,sizeof(contenido));

while(cantidad=read(sd,buff,tam)>0)
{
printf("%s");
}

close(sd);

return 1;
}
And I still get a error message:

Conexion establecida
HTTP/1.1 400 Bad Request
Date: Mon, 05 May 2008 13:56:49 GMT
Server: Apache/2.2.8 (Unix)
Content-Length: 292
Connection: close
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
Request header field is missing ':' separator.<br />
<pre>
</pre>
</p>
</body></html>
ser sent a request that this server could not understand.<br />
Request header field is missing ':' separator.<br />
<pre>
</pre>

didi · #3 Post by **didi** » 2008-05-05 14:38

What happens if you replace \n with \r\n ?

Some time ago I made a WebClient with COM/ATL and here is how I serialized the HTTP Headers set to it:

Code: Select all

STDMETHODIMP CHttpHeaders::toString(BSTR *pVal)
{
	CStdString newLine = _T("\r\n");
	CStdString RetVal;

	Lock();
	ContainerType::iterator it = m_coll.begin();
	while ( it != m_coll.end() ) {
		CComPtr<IHttpHeader> pHeader;

		pHeader = it->second;
		BSTR				strKey;
		BSTR				strValue;
		pHeader->get_Name( &strKey);
		pHeader->get_Value( &strValue);
		RetVal += CStdString(strKey) + _T(": ") + CStdString(strValue) + newLine;
		it++;
	}

	*pVal = CComBSTR(RetVal).Detach();
	Unlock();
	return S_OK;
}

debianSuiz · #4 Post by **debianSuiz** » 2008-05-05 14:47

I do what you said but I still get this error and I don'nt understand your code. thanks anyway didi!!

didi · #5 Post by **didi** » 2008-05-05 15:43

It's been a while for me too. It was part of a project which allowed me to construct my own request, but it uses the Win32 API, so probably not what you want

Also iirc, there should be some kind of separator between the headers and the body. I think it's a double \r\n, but you should inspect the relevant RFC for that.
EDIT: nvm, you're making a GET request, so there is no body

What you can do is install the LiveHTTPHeaders add-on for Firefox/Iceweasel, which allows you to inspect the headers send and received. Maybe that'll help you tackle the problem.

And as far as I know (but it's been a while) you are not required to send all those headers, just use the ones you really need. Less headers -> less chances for errors.

Just noticed a typo in one of your headers:
Accept-Enconding should be Accept-Encoding

Good luck

debianSuiz · #6 Post by **debianSuiz** » 2008-05-06 14:03

Hello didi. I found a good code that works. I show you if you interest.

#include <stdio.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <sys/types.h>
#include <unistd.h>
#include <arpa/inet.h>
#include <string.h>
#define tam 90
#define tam_recv 1
#define tam_desp 256
#define port 80
#define host "192.168.0.2"
//#define host "127.0.0.1"

int main(void)
{
FILE *f;
int sd,newsd,error,cantidad;
char buff[tam];
char buff_recv[tam_recv];
char buff_desp[tam_desp];
char contenido[]="";
sockaddr_in serveraddr;
f=fopen("imagen.jpg","wb");
sd=socket(AF_INET,SOCK_STREAM,0);

if(sd==-1)
printf("Error al crear socket\n");

serveraddr.sin_family=AF_INET;
serveraddr.sin_addr.s_addr=inet_addr(host);
serveraddr.sin_port=htons(port);
error=connect(sd,(struct sockaddr*)&serveraddr,sizeof(serveraddr));
if(error<0)
{
printf("Error al conectarse\n");
}
else printf("Conexion establecida\n");

strcpy(buff,"GET /cgi-bin/video.jpg HTTP/1.1\r\nHost: 192.168.0.2\r\nConnection: close\r\n\r\n");
write(sd,buff,sizeof(buff));
write(sd,contenido,sizeof(contenido));

read(sd,buff_desp,tam_desp);

while(cantidad=read(sd,buff_recv,tam_recv)>0)
{
fwrite(buff_recv,sizeof(char),tam_recv,f);
}
fclose(f);
close(sd);

return 1;
}

There is only a question. You can see, I define the size of receive at one byte. I have to do this because of if I set the size at 500 bytes for example, I get a corrupt image, with colours wrong. Do you know why this happens?
I'm sorry because my english is not very well. If you don't understand me, please said me it. Thank you very much didi!!!

MarkG · #7 Post by **MarkG** » 2008-05-06 15:21

debianSuiz wrote:I have to do this because of if I set the size at 500 bytes for example, I get a corrupt image, with colours wrong. Do you know why this happens?

It may be because you're apparently ignoring the size value returned by the read call. I'm not sure whether your socket will block until it has 500 bytes, or just return whatever is available; for safety, you should be writing out the number of bytes it actually read, not the number in the buffer.

Damotclese · #8 Post by **Damotclese** » 2008-05-28 15:23

You can download the source code to NOTServer at http://www.notserver.com/ and see how the HTTP GET request is formed there.

djtarki · #9 Post by **djtarki** » 2010-09-07 10:18

======SOLVED======

It was a problem of the header size after all . I needed to read 170 bytes instead of 168.

This is the working code (just in case you need it)

Code: Select all

#include <stdio.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <sys/types.h>
#include <unistd.h>
#include <arpa/inet.h>
#include <string.h>

//Size of command buffer. It must be big enough to store the header request command.
#define tam 450U

//When receiving the image
#define tam_recv 1U

#define tam_desp 170U //Response seems to be Ok

#define port 80

//Camera IP
#define host "172.29.33.161"

int main(void)
{
FILE *f;
char key = 0;
int sd,error,cantidad;
char buff[tam];
memset(buff, 0, tam);
char buff_recv[tam];
char buff_desp[tam_desp];
char contenido[]="";
sockaddr_in serveraddr;
sd=socket(AF_INET,SOCK_STREAM,0);

if(sd==-1)
{
	 perror("Socket error");
	return -1;
}

serveraddr.sin_family=AF_INET;
serveraddr.sin_addr.s_addr=inet_addr(host);
serveraddr.sin_port=htons(port);

error=connect(sd,(struct sockaddr*)&serveraddr,sizeof(serveraddr));
if(error<0)
{
	perror("> Connection error");
	close(sd);
	return -1;
}
else 
 printf("> Connection granteed\n\n");

while(key!='q')
{

strcpy(buff,"GET /axis-cgi/jpg/image.cgi?resolution=640x480 HTTP/1.1\r\nHost: 172.29.33.161\r\nUser-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.8) Gecko/20100723 Ubuntu/10.04 (lucid) Firefox/3.6.8\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip,deflate\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nKeep-Alive: 115\r\nConnection: keep-alive\r\nCache-Control: max-age=0\r\n\r\n");


/*GET /axis-cgi/jpg/image.cgi?resolution=640x480 HTTP/1.1*/
//Host: 172.29.33.161
//User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.8) Gecko/20100723 Ubuntu/10.04 (lucid) Firefox/3.6.8
//Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*;q=0.8
//Accept-Language: en-us,en;q=0.5
//Accept-Encoding: gzip,deflate
//Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
//Keep-Alive: 115
//Connection: keep-alive
//Cache-Control: max-age=0


#ifdef _DEBUG
	//Print request
	printf("%s", buff);
#endif

write(sd,buff,tam);
write(sd,contenido,sizeof(contenido));

//Read the header only to know if the answer is Ok
cantidad = read(sd,buff_desp,tam_desp);
//Print the response
#ifdef _DEBUG
	printf("%s",buff_desp);
#endif

f=fopen("received_image","wb");
while((cantidad=read(sd,buff_recv, tam_recv))>0)
{
  fwrite(buff_recv,sizeof(char),tam_recv,f);
}
fclose(f);

printf("\n> Press any key to continue or 'q' to exit ");
key=getchar();

}

//Close sockect descriptor
close(sd);

return 1;
}

Regards.

Hello everyone,

I found your answers very useful, thanks. However I still cannot receive the image properly. When I try to open the image I get "Error interpreting JPEG image file (Not a JPEG file: starts with 0x0d 0x0a". In addition the size of the file doesn't fit with the one of the server response: 71854 the downloaded file VS 71852 of the server response).

Here is the modified code I've compiled and executed:

Code: Select all

#include <stdio.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <sys/types.h>
#include <unistd.h>
#include <arpa/inet.h>
#include <string.h>
//Size of command buffer. It must be  big enough to store the header request command.
#define tam 500

//When receiving the image
#define tam_recv 1

//#define tam_desp 500
#define tam_desp 168 //Response seems to be Ok
//#define tam_desp 160
//#define tam_desp 141

#define port 80
#define host "172.29.33.161" //My network camera address

int main(void)
{
FILE *f;
int sd,error,cantidad;
char buff[tam];
memset(buff, 0, tam);
char buff_recv[tam_recv];
char buff_desp[tam_desp];
char contenido[]="";
sockaddr_in serveraddr;
f=fopen("imagen.jpg","wb");
sd=socket(AF_INET,SOCK_STREAM,0);

if(sd==-1)
printf("Error al crear socket\n");

serveraddr.sin_family=AF_INET;
serveraddr.sin_addr.s_addr=inet_addr(host);
serveraddr.sin_port=htons(port);
error=connect(sd,(struct sockaddr*)&serveraddr,sizeof(serveraddr));
if(error<0)
{
printf("Error al conectarse\n");
}
else 
 printf(">Conexion establecida\n\n");

strcpy(buff,"GET /axis-cgi/jpg/image.cgi?resolution=640x480 HTTP/1.1\r\nHost: 172.29.33.161\r\nUser-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.8) Gecko/20100723 Ubuntu/10.04 (lucid) Firefox/3.6.8\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip,deflate\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nKeep-Alive: 115\r\nConnection: keep-alive\r\nCache-Control: max-age=0\r\n\r\n");

The previous line is based on what I see in Live HTTP Headers add-on for Firefox:

/*GET /axis-cgi/jpg/image.cgi?resolution=640x480 HTTP/1.1*/

//Host: 172.29.33.161

//User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.8) Gecko/20100723 Ubuntu/10.04 (lucid) Firefox/3.6.8

//Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*;q=0.8

//Accept-Language: en-us,en;q=0.5

//Accept-Encoding: gzip,deflate

//Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

//Keep-Alive: 115

//Connection: keep-alive

//Cache-Control: max-age=0


//Print request to test everything is Ok
printf("%s", buff);

write(sd,buff,sizeof(buff));
write(sd,contenido,sizeof(contenido));

read(sd,buff_desp,tam_desp);

//Print response to see everything is Ok
printf("%s",buff_desp);

while((cantidad=read(sd,buff_recv,tam_recv))>0)
{
fwrite(buff_recv,sizeof(char),tam_recv,f);
}

fclose(f);

//Close sockect descriptor
close(sd);

return 1;
}

When I use the browser and check what it's received with Live HTTP Headers:

/*HTTP/1.0 200 OK*/
/*Cache-Control: no-cache*/
/*Pragma: no-cache*/
/*Expires: Thu, 01 Dec 1994 16:00:00 GMT*/
/*Connection: close*/
/*Content-Type: image/jpeg*/
/*Content-Length: 71852*/

And what I see after my program prints the info is:

HTTP/1.0 200 OK
Cache-Control: no-cache
Pragma: no-cache
Expires: Thu, 01 Dec 1994 16:00:00 GMT
Connection: close
Content-Type: image/jpeg
Content-Length: 71852
� ->> What is this?

Any idea of what am I doing wrong?

Thanks in advance.

Regards.

Debian User Forums