shutil.copy corrupting files?




Has anyone here ever come across the shutil copy method corrupting files (on Linux)

I've just finished writing my first Python script for use to help me automate an installation process I have to do fairly regularly, but it is giving me some grief!

The script basically copies some files into a virtual site on a server for use in an eCommerce application and does the following:
1 - lets me select the site for the installation
2 - connects via FTP to a server that holds the required files - using the ftplib module and the FTP class
3 - downloads a text-file containing a list of the files and the deed destination paths for these files. This text file is parsed and the required files are downloaded to a temp location. Any destination paths that don't already exist are created in the virtual site's filesystem using os.makedir
4 - the files are then copied from the temp location into the virtual site using shutil.copy
5 - finally the correct ownerships and permissions are applied to the new files.
6 - the software installation is then completed manually through a browser (using the eCommerce software)

Unfortunately, when I come to add the software to the site using the eComm software, they are causing internal server errors on the web server (giving "premature end of script headers" in the error logs - something I usually see from corrupt scripts). I know the files are ok to begin with, as going through the above process manually does not cause the web server to choke.

At first I assumed the FTP transfer must have been trashing the files, so I tried commenting that part out of the script and just copying local versions of the files into the website using shutil.copy instead. This didn't help. It looks to me like shutil.copy is the culprit. I'd love to know if anyone else has come across this, I haven't been able to find anything while googling though, so it doesn't look too hopeful.

Does anyone have any tips on how else I could find what's causing the trouble

Interstingly, when I compare the source files to my copies, they have exactly the same filesizes. Unfortunately, they are data files, so I can't view the contents to look for problems.

Posted On: Friday 28th of December 2012 05:27:40 AM Total Views:  169
View Complete with Replies

RELATED TOPICS OF Python Programming PROGRAMMING LANGUAGE




PyCon 2007 dates announced

The planning committee has set the date for the next PyCon conference: PyCon 2007 will be February 23 to 25 2007, in Addison, Texas (same location as 2006). Here's the schedule: Tutorial Day Thu Feb 22, 2007 Conference Day #1 Fri Feb 23, 2007 Conference Day #2 Sat Feb 24, 2007 Conference Day #3 Sun Feb 25, 2007 Sprint Day #1 Mon Feb 26, 2007 Sprint Day #2 Tue Feb 27, 2007 Sprint Day #3 Wed Feb 28, 2007 Sprint Day #4 Thu Mar 01, 2007 Andrew M. Kuchling amk@amk.ca Co-chair, PyCon 2007 http://us.pycon.org
VIEWS ON THIS POST

106

Posted on:

Monday 5th November 2012
View Replies!

collections.chain

Several languages like Java, C# etc have a List type in the std lib. Python has a built-in list(), it's implemented as array dynamic on the right. Not too much time ago Hettinger has added a collections.deque (its C code is really nice), that compared to list() allows a faster append on the right and a much faster prepend on the left. It's implemented as a double linked list of fixed-sized blocks. blocks but the first and last are fully filled, so such blocks don't need to store their length, and the data structure just needs to store the length of the first and last block. In the C++ STL I think the deque can be implemented as a dynamic array of pointers, that point to the start of each fixed-size block. This allows a faster access of items, because you need two lookups and you don't need to follow the linked list (plus maybe a module operation). I don't know why the collections.deque uses a double linked list, maybe because it allows a simpler design (in the dynamic arrays of pointers you have to manage them as a circular array, so you need the module or an if). A double-ended queue covers lot of usages of a linked list, but not all of them. So if enough Python programmers feel the need of a (light) data structure that allows O(1) removal and add of items in any point, then such data structure can be created. The name can be "chain", because it's easy, short, it means the right thing, and "list" is already taken. Its implementation can be a normal double linked list. But on modern CPUs they can be not much efficient, so there are few strategies to improve that: http://en.wikipedia.org/wiki/CDR_coding http://en.wikipedia.org/wiki/Unrolled_linked_list I think an unrolled double linked list is a fit implementation for this purpose. This data structure is quite similar to the collections.deque, but each block has to keep the number of items it contains (note: if experiments show that such overhead in memory and speed is little enough, then most of the C code of the deque may even be thrown away, using the chain to implement it). Are enough Python programmers interested in such chain data structure Can the typical Python programs enjoy the use of it I presume it's not very important, but sometimes I have found a use for it. If enough people are interested in this data structure, then I think there's a problem to be solved. How to manage references into the chain itself. You need references, otherwise many operations become O(n), and this makes the chain quite less useful. A simple solution is to create another independent object like chainptr, that's essentially a pointer to an item of the chain, it can also become nil/Nil/Null/null, I presume... If you have two chainptr that point the n-th item, and you remove the n-th item, then the second chainptr now doesn't point to the n+1-th item (this is true for python list, where pointers are integer numbers), it's a broken pointer. Etc. Such low-level problems are probably out of place in Python. A way to avoid those problems is to make the pointers part of the chain object itself, so they are kept consistent. I presume there must be some way to add/remove such indexes dynamically, but I presume in Python this is not too much a problem, but such kind of management slows down the data structure a little. Every operation has to control the state of all defined indexes, but I presume their number is usally low (one, two) so it may not be a problem. I don't have ideas for a good API yet, but if the pointers are part of the chain, the syntax may becomes something like: from collections import chain d = chain("ABCD") d.addindex() # creates p0 that points A d.p0.next() d.p0.next() # now p0 point C d.addindex(d.p0) # now p1 point C d.p0.delete() # now p0 and p1 point D (or nothing) Well, that's ugly. But first of all it's important to see if a chain i useful, if the answer is positive, then I can look for a decent API. Bye, bearophile
VIEWS ON THIS POST

27

Posted on:

Wednesday 7th November 2012
View Replies!

random.random(), random not defined!?

do i need to import something to use random...
VIEWS ON THIS POST

51

Posted on:

Saturday 10th November 2012
View Replies!

Re: More Fun With MySQL and Images

On Thu, 2008-04-17 at 09:52 -0500, Victor Subervi wrote: > Never mind. Apparently, these tags throw it for that loop: > print '\n' > I´m surprised they would, but gratified I found the problem. > Victor > > Why does ...
VIEWS ON THIS POST

49

Posted on:

Saturday 10th November 2012
View Replies!

ANN: PyTables 1.2

PyTables is a library to deal with very large datasets. It leverages the excellent HDF5 and numarray libraries to allow doing that in a very efficient way using the Python language. More info in: http://pytables.sourceforge.net/ ========================= Announcing PyTables 1.2 ========================= ...
VIEWS ON THIS POST

17

Posted on:

Saturday 10th November 2012
View Replies!

RELEASED Python 2.4.2 (final)

On behalf of the Python development team and the Python community, I'm happy to announce the release of Python 2.4.2 (final). Python 2.4.2 is a bug-fix release. See the release notes at the website (also available as Misc/NEWS in the ...
VIEWS ON THIS POST

47

Posted on:

Saturday 10th November 2012
View Replies!

Re: Parsing HTML?

On Wed, 2008-04-02 at 21:59 -0700, Benjamin wrote: > I'm trying to parse an HTML file. I want to retrieve all of the text > inside a certain tag that I find with XPath. The DOM seems to make > ...
VIEWS ON THIS POST

31

Posted on:

Saturday 10th November 2012
View Replies!

SC-Track Roundup 0.7.6 - an issue tracking system

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Roundup is a simple-to-use and -install issue-tracking system with command-line, web and e-mail interfaces. It is based on the winning design from Ka-Ping Yee in the Software Carpentry "Track" design competition. Roundup now has ...
VIEWS ON THIS POST

47

Posted on:

Saturday 10th November 2012
View Replies!

simple web-server

hello python users, could you point me to a very simple (single file is best) web-server I want to serve a few web-forms and run some shell scripts when the forms are submitted. I might add Ajax later (this is ...
VIEWS ON THIS POST

47

Posted on:

Saturday 10th November 2012
View Replies!

Python 3000 and import __hello__

Just playing around with Python3000 a2 release on Windows XP 32-bit x86. import __hello__ doesn't print 'hello world...' as it does on 2.5 The import doesn't fail or generate errors... just no output. Perhaps this is by design Brad...
VIEWS ON THIS POST

47

Posted on:

Saturday 10th November 2012
View Replies!

Movie Stars As Sources Of Wisdom

Why do many people look to movie stars for answers to some of life's most challenging questions While we have great respect for the art of acting, as explicated from Stanislavsky to Strasberg, the latter of whom we knew well ...
VIEWS ON THIS POST

22

Posted on:

Saturday 10th November 2012
View Replies!

simple string formatting question

I have a list of strings (sys.argv actually). I want to print them as a space-delimited string (actually, the same way they went into the command line, so I can cut and paste) So if I run my program like: ...
VIEWS ON THIS POST

50

Posted on:

Saturday 10th November 2012
View Replies!

Re: sorting contacts alphabetically, sorted by surname

Marie Hughes wrote: > HI > > I have to write a program that contains a text file in the following format: > > Academic Position Room Ext. Email > Prof Marie Maguire Head of > School M9002 0000 lp.muire@ulioooor.ac.uk ...
VIEWS ON THIS POST

56

Posted on:

Saturday 10th November 2012
View Replies!

RELEASED Python 3.0rc2

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On behalf of the Python development team and the Python community, I am happy to announce the second release candidate for Python 3.0. This is a release candidate, so while it is not suitable ...
VIEWS ON THIS POST

30

Posted on:

Saturday 10th November 2012
View Replies!

length of a tuple or a list containing only one element

Hi everybody, I have a question about the difference of behavior of "len" when applied on tuples or on lists. I mean: $ len( ( 'foo', 'bar' ) ) 2 $ len( ( 'foo' ) ) 3 $ len( [ ...
VIEWS ON THIS POST

39

Posted on:

Saturday 10th November 2012
View Replies!

Understanding this generator function

, This is an example of a generator function: ===== def counter(start_at=0): count = start_at while True: val = (yield count) if val is not None: count = val else: count += 1 ====== >>> count = counter(5) >>> count.next() ...
VIEWS ON THIS POST

46

Posted on:

Sunday 11th November 2012
View Replies!

passing arguments to exec

i want to execute a python script using exec open('script.py'). how do I pass arguments...
VIEWS ON THIS POST

57

Posted on:

Sunday 11th November 2012
View Replies!

Re: Win32 trouble with threading, signals, and sleep()

Lowell eman wrote: > I'm running into this issue on Windows with the follow exception at > the time when the signal handler is called: > > Traceback (most recent call last): > ... > self.done.wait(30) > File "D:\Python24\lib\threading.py", line ...
VIEWS ON THIS POST

71

Posted on:

Sunday 11th November 2012
View Replies!

Re: regex doubts

Mr SZ wrote: > I am taking a string as an input from the user and it should only > contain the chars:L , M or R > > I tried the folllowing in kodos but they are still not ...
VIEWS ON THIS POST

79

Posted on:

Sunday 11th November 2012
View Replies!

PyOhio registration is open

Registration is open for PyOhio, the daylong regional miniconference on July 26 in Columbus, OH. http://www.pyohio.org/reg/register/ Registration is free, but registering early guarantees your spot and helps the organizers schedule talks to avoid conflicts. See you on July 26 in ...
VIEWS ON THIS POST

138

Posted on:

Sunday 11th November 2012
View Replies!

struct.pack behavior

Can anyone explain to me why struct.pack('HB',1,2) gives 3 bytes, whereas struct.pack('BH',1,2) gives 4 bytes -Steven...
VIEWS ON THIS POST

33

Posted on:

Sunday 11th November 2012
View Replies!

Re: The best way to package a Python module?

On Mon, 16 Jun 2008 08:39:52 +1000, Ben Finney wrote: >Jean-Paul Calderone writes: > >> On Mon, 16 Jun 2008 01:37:47 +0900, js wrote: >> >By "package", I meant APT, Ports for BSD, MacPorts, etc. >> >> ...
VIEWS ON THIS POST

45

Posted on:

Sunday 11th November 2012
View Replies!

Re: investigate python auth problem

David Hláik wrote: > > I have reproduced steps, to show you sample on another module and its > results in INN (becouse i really like to solve this Since I don't see anything related to python-ldap please don't follow-up ...
VIEWS ON THIS POST

49

Posted on:

Sunday 11th November 2012
View Replies!

Re: Removing Space and "-" from a string

Shakir, > I have thousands of records in MS Access database table, which records I > am fetching using python script. One of the columns having string like > '8 58-2155-58' > > Deed output: '858215558' > > I want ...
VIEWS ON THIS POST

48

Posted on:

Sunday 11th November 2012
View Replies!

kate hudson wallpapers

Just few link on some Movies Free Movies: http://exclusive.12w.net F R E E C E L E B R I T Y M O V I E S...
VIEWS ON THIS POST

61

Posted on:

Sunday 11th November 2012
View Replies!

Python - CGI - XML - XSD

Hi , Quick question. I've got an XML schema file (XSD) that I've written, that works fine when my data is present as an XML file. (Served out by apache2.) Now when I call python as a cgi script, and ...
VIEWS ON THIS POST

22

Posted on:

Sunday 11th November 2012
View Replies!

Minimalistic Software Transactional Memory

I'm interested in writing a simple, minimalistic, non persistent (at this stage) software transactional memory (STM) module. The idea being it should be possible to write such a beast in a way that can be made threadsafe fair easily. ...
VIEWS ON THIS POST

43

Posted on:

Sunday 11th November 2012
View Replies!

Modules and descriptors

As I understand it, the appeal of properties (and descriptors in general) in new-style classes is that they provide a way to "intercept" direct attribute accesses. This lets us write more clear and concise code that accesses members directly without ...
VIEWS ON THIS POST

21

Posted on:

Monday 12th November 2012
View Replies!

ctypes

I am trying to call a funtinon in a third party dll that spawns another exe and I am using ctypes. Python does not complain at all but the other process does not get spawned. It appears that I am ...
VIEWS ON THIS POST

26

Posted on:

Monday 12th November 2012
View Replies!

Re: Network-Packets

-On [20080102 16:00], Sunil Ghai (sunilkrghai@gmail.com) wrote: >I know this is not the right place for asking about this but i am sure some of >you must have an idea about this. The networking community would be more appropriate, methinks. ...
VIEWS ON THIS POST

21

Posted on:

Monday 12th November 2012
View Replies!

connect to server question

Is there a way to connect to servers on OS X using Python Here's the code I presently use in a Bash Shell script: /bin/mkdir -p /Volumes/A_Share /sbin/mount_afp "afp://username assword@ServerBox/A_Share" "/ Volumes/A_Share" Can something similar be done with Python
VIEWS ON THIS POST

36

Posted on:

Monday 12th November 2012
View Replies!

looking for ocbc example

I am sure this is what I want: http://www.python.org/topics/database/docs.html "The documentation for the PythonWin ODBC module." but it is 404. google isn't being nice. Anyone know where I can find some simple examples I have used odbc in other environments, ...
VIEWS ON THIS POST

85

Posted on:

Monday 12th November 2012
View Replies!

Finding lowest value in dictionary of objects, how?

I'm new to Python and working on a school assignment. I have setup a dictionary where the keys point to an object. Each object has two member variables. I need to find the smallest value contained in this group ...
VIEWS ON THIS POST

21

Posted on:

Monday 12th November 2012
View Replies!

Re: When I do from lxml import etree I've this error : AttributeError:'module' object has no attribute 'BytesIO'

Le Mon, 25 Aug 2008 13:50:50 +0000, KLEIN Stéphane a écrit : > > > I'm on Ubuntu 8.04.1 > > I've installed lxml with easy_install lxml command. > > Now, when I load etree I've this error : > ...
VIEWS ON THIS POST

73

Posted on:

Sunday 25th November 2012
View Replies!

You advice please

, I'm a Java/Java EE developer and I'm playing with Python these days. I like the Python language so much and I like its communities and the Django framework. My friends are about to open a Ruby/Rails shop and they ...
VIEWS ON THIS POST

74

Posted on:

Sunday 25th November 2012
View Replies!

Simple Doc Test failing without any reason [Help Needed]

The following following code fails with the failiure:- File "test.py", line 27, in __main__.sanitize_number Failed example: sanitize_number('0321-4683113') Expected: '03214683113' Got: '03214683113' Expected and Got looks the same. The value should verify. What am I doing wrong here
VIEWS ON THIS POST

53

Posted on:

Sunday 25th November 2012
View Replies!

simple way to touch a file if it does not exist

what are the simple ways I could think of os.open(), os.exec(touch file) are there any simpler methods...
VIEWS ON THIS POST

49

Posted on:

Sunday 25th November 2012
View Replies!

Python module for reading FilePro files?

Does anyone know of a Python package or module to read data files from the venerable old Filepro crossplatform database/IDE...
VIEWS ON THIS POST

66

Posted on:

Sunday 25th November 2012
View Replies!

Re: Search the command history - Python Shell

> Are there any simillar key combination in Python Shell like Linux Ctrl+R > (reverse-i-search) to search the command history It must depend on how your version of Python was built...mine here on my Linux box has exactly that functionality. ...
VIEWS ON THIS POST

42

Posted on:

Sunday 25th November 2012
View Replies!

Re: wxEVT_SCROLL_ENDSCROLL

try wx.EVT_SCROLL_ENDSCROLL Jack Holt wrote: > , > > I got the following error: > > Traceback (most recent call last): > File "vplayer.py", line 15, in > File "config.pyo", line 3, in > File "wx\__init__.pyo", line ...
VIEWS ON THIS POST

44

Posted on:

Sunday 25th November 2012
View Replies!

Re: Fw: Undeliverable Message

> Heres the code > > def increment(digits,symbols): > overflow=True > digitpos=-1 > while overflow and -digitpos<=len(digits): > digitsymbolindex=symbols.index(digits[digitpos]) > ...
VIEWS ON THIS POST

34

Posted on:

Sunday 25th November 2012
View Replies!

SUMMON 1.8.4 RELEASED: rapid prototyping of 2D visualizations

SUMMON 1.8.4 SUMMON is a python extension module that provides rapid prototyping of 2D visualizations. By heavily relying on the python scripting language, SUMMON allows the user to rapidly prototype a custom visualization for their data, without the overhead of ...
VIEWS ON THIS POST

58

Posted on:

Sunday 25th November 2012
View Replies!

[help request] how to set sys.stderr to object of cStringIO type

I am in a perculiar situation. I want to use PyRun_SimpleString for creating Python functions in embedded Python in C++. But there could be cases when Python function code compilation could fail & PyRun_SimpleString will return -1 as return status. ...
VIEWS ON THIS POST

100

Posted on:

Sunday 25th November 2012
View Replies!

gtk.TreeView cell inconsistent state

In the below code setting cell to inconsistent sets entire column inconsistent (renderer).However, I need a third state off | on | inconsistent . How can I change one path->cell
VIEWS ON THIS POST

58

Posted on:

Monday 26th November 2012
View Replies!

Re: help displaying pdf thru client/server

On Wed, 19 Dec 2007 13:50:10 -0600, Larry Bates wrote: >PaulS wrote: >> Connecting to a Linux server from XP pc using a telnet program, I run a >> report and convert it to a pdf document(using Reportlab) which ...
VIEWS ON THIS POST

43

Posted on:

Monday 26th November 2012
View Replies!

My pyguessinggame

Could someone look this over (Toggle Plain Text) #import whrandom def start(): answer=randint(1,101) #the answer guessnum=0 #number of user guesses guess=raw_imput ('guess my number, it is between 1 and 100') #user's guess while guess!= answer: guessnum=guessnum+1 if(guess = 101): print'no the answer is below 101' elif (guess > answer): print 'Too high' elif (guess answer): print 'Too high' elif (guess
VIEWS ON THIS POST

41

Posted on:

Tuesday 27th November 2012
View Replies!

Function won't exit on return!

(Toggle Plain Text) # def find_items(url): # try: # url = urllib.urlopen(url).read() # except: # return None # while True: # web_page = 'some url' # fs_item = url.find(r'some text') # fs_item += 9 # fe_item= url.find(r'"',fs_item) # item.append(web_page[:]+url[fs_item:fe_item]) # ps_s = url.find(r'"some text">Next') # if ps_s is -1: no_next = 1 # else: no_next = 0 # if not no_next: # ps_s-=25 # ps_e = url.find(r'"',ps_s) # next_page = '' # next_page= next_page + web_page[:] + url[ps_s:ps_e] # for items in range(1,25): # ss_item = url.find(r'some text',fe_item) # if ss_item is -1: break # ss_value = ss_item # char_to_kill = url[ss_item-1:ss_item] # if url[ss_item-1:ss_item] is char_to_kill: ss_item = url.find(r'some text'ss_value+9) # if ss_item is -1: # if no_next is 1: # return item #
VIEWS ON THIS POST

31

Posted on:

Wednesday 28th November 2012
View Replies!

What's with the baggage? py2exe/compilers

Hi people, i've just been reading some of the threads about the py2exe thing. everytime someone asks about turning their .py into .exe this comes up and they say that it actualy packs the python.exe and other stuff into the executable... isn't that unnecessary I dont know too much about this (only started learning python and Java recently) but i'm sure thats not how it would be done professioanlly, like with the day-today exec.s that uses. Is there actually a way to turn code into executable, or have i missed something BTW, how hard is it to advance from console programming to GUI programming, in python
VIEWS ON THIS POST

56

Posted on:

Friday 28th December 2012
View Replies!

Calculating

I will can't at this second at library but, when I get home I will hey you got any ideas. I have ubuntu but, when I save a file as .py now it opens up a text editor how can i change this
VIEWS ON THIS POST

39

Posted on:

Friday 28th December 2012
View Replies!

I'm a new to Python

, nice forums you have here. Anyway I've started to learn python, today infact, as I've heard its a good language to learn, I have some experience in BASIC and a little in C from using Linux. I was wondering if there was a function in Python that did the same as mid$() in BASIC, I've searched for tutorials on string manipulation etc, but could not find anything on this function.
VIEWS ON THIS POST

235

Posted on:

Friday 28th December 2012
View Replies!