[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[no subject]



<a  rel="nofollow" href="http://elonen.iki.fi/code/misc-notes/remove-duplicate-files/";>http://elonen.iki.fi/code/misc-notes/remove-duplicate-files/</a>

On Sunday 06 February 2005 12:37, pras at cycloeastern.com wrote:
&gt; Only couple of enhancements I would make is 
&gt; 
&gt; make ls --sort...
&gt; to ls -ltr --sort as he said the pics might be nested in a directory.
&gt; 
&gt; Also
&gt; 
&gt; I would do 'chksum' and match that as well before the rm..
&gt; 
&gt; Just my 2 cents
&gt; -Prasanna Subash
&gt; 
&gt; 
&gt; On Sun, Feb 06, 2005 at 11:51:23AM -0500, Jim Patterson wrote:
&gt; &gt; Trey,
&gt; &gt; 
&gt; &gt; The following script is primitive, but I think it should work fine for 
you.
&gt; &gt; If you name if del_dup.sh, then run it with:
&gt; &gt; ./del_dup.sh /photos /photos/* /photos/*/*
&gt; &gt; 
&gt; &gt; It does not recurse automagicaly, but you can extend it with too much
&gt; &gt; work.  Just take the top level directory and use find to locate all the
&gt; &gt; sub-directories.  Then pass that list to ls.  NOTE: you can not use
&gt; &gt; ls -R since that breaks each directory into seperate sections.
&gt; &gt; 
&gt; &gt; Jim P.
&gt; &gt; 
&gt; &gt; #! /bin/sh
&gt; &gt; 
&gt; &gt; oldsize=&quot;-&quot;
&gt; &gt; oldfile=&quot;-&quot;
&gt; &gt; 
&gt; &gt; for f in `ls --sort=size $*`
&gt; &gt; do
&gt; &gt;     dl=`ls -l &quot;$f&quot; | tr -s &quot; &quot;`
&gt; &gt;     size=`echo $dl | cut -f 5 -d &quot; &quot;`
&gt; &gt;     file=`echo $dl | cut -f 9 -d &quot; &quot;`
&gt; &gt; 
&gt; &gt;     if [ &quot;$oldsize&quot; = &quot;$size&quot; ]
&gt; &gt;     then
&gt; &gt;         if diff -q &quot;$oldfile&quot; &quot;$file&quot; &gt;/dev/null
&gt; &gt;         then
&gt; &gt;             echo Deleting $file dup of $oldfile
&gt; &gt;             rm &quot;$file&quot;
&gt; &gt;         fi
&gt; &gt;     else
&gt; &gt;         oldsize=$size
&gt; &gt;         oldfile=$file
&gt; &gt;     fi
&gt; &gt; done
&gt; &gt; 
&gt; &gt; On Sun, 06 Feb 2005 11:19:42 -0500, Trey Sizemore &lt;trey at fastmail.fm&gt; 
wrote:
&gt; &gt; &gt; I've got a number of photos from my digital camera stored on my machine
&gt; &gt; &gt; and a number of them are duplicates.  I was wondering if there was a
&gt; &gt; &gt; script that would identify the duplicates and their locations so I can
&gt; &gt; &gt; purge them to better organize and post.  There is a /photos directory
&gt; &gt; &gt; that itself contains both photos and other sub-directories so it would
&gt; &gt; &gt; need to search both the /photos directory and all sub-directories within
&gt; &gt; &gt; it.
&gt; &gt; &gt; 
&gt; &gt; &gt; Am I wishfully thinking?  I don't have experience with sed and awk
&gt; &gt; &gt; (although I've used grep and pipes in the past) but this is beyond my
&gt; &gt; &gt; limited understanding.
&gt; &gt; &gt; 
&gt; &gt; &gt; Thanks.
&gt; &gt; &gt; 
&gt; &gt; &gt; --
&gt; &gt; &gt; Cheers,
&gt; &gt; &gt; Trey
&gt; &gt; &gt; ---
&gt; &gt; &gt; 
&gt; &gt; &gt; &quot;Men do not differ much about what things they will call evils;
&gt; &gt; &gt; they differ enormously about what evils they will call excusable.&quot;
&gt; &gt; &gt; -- GK Chesterton
&gt; &gt; &gt; 
&gt; &gt; &gt; 11:15AM up 3:03, 0 users, load averages: 0.39, 0.43, 0.61
&gt; &gt; &gt; FreeBSD salamander.thesizemores.net 5.3-STABLE i386
&gt; &gt; &gt; 
&gt; &gt; &gt; _______________________________________________
&gt; &gt; &gt; Ale mailing list
&gt; &gt; &gt; Ale at ale.org
&gt; &gt; &gt; <a  rel="nofollow" href="http://www.ale.org/mailman/listinfo/ale";>http://www.ale.org/mailman/listinfo/ale</a>
&gt; &gt; &gt;
&gt; &gt; _______________________________________________
&gt; &gt; Ale mailing list
&gt; &gt; Ale at ale.org
&gt; &gt; <a  rel="nofollow" href="http://www.ale.org/mailman/listinfo/ale";>http://www.ale.org/mailman/listinfo/ale</a>
&gt; _______________________________________________
&gt; Ale mailing list
&gt; Ale at ale.org
&gt; <a  rel="nofollow" href="http://www.ale.org/mailman/listinfo/ale";>http://www.ale.org/mailman/listinfo/ale</a>
&gt; 


</pre>
<!--X-Body-of-Message-End-->
<!--X-MsgBody-End-->
<!--X-Follow-Ups-->
<hr>
<ul><li><strong>Follow-Ups</strong>:
<ul>
<li><strong><a name="00185" href="msg00185.html">[ale] Looking for duplicate photos</a></strong>
<ul><li><em>From:</em> trey at fastmail.fm (Trey Sizemore)</li></ul></li>
</ul></li></ul>
<!--X-Follow-Ups-End-->
<!--X-References-->
<ul><li><strong>References</strong>:
<ul>
<li><strong><a name="00179" href="msg00179.html">[ale] Looking for duplicate photos</a></strong>
<ul><li><em>From:</em> trey at fastmail.fm (Trey Sizemore)</li></ul></li>
<li><strong><a name="00180" href="msg00180.html">[ale] Looking for duplicate photos</a></strong>
<ul><li><em>From:</em> unixdude at gmail.com (Jim Patterson)</li></ul></li>
<li><strong><a name="00182" href="msg00182.html">[ale] Looking for duplicate photos</a></strong>
<ul><li><em>From:</em> pras at cycloeastern.com (pras at cycloeastern.com)</li></ul></li>
</ul></li></ul>
<!--X-References-End-->
<!--X-BotPNI-->
<ul>
<li>Prev by Date:
<strong><a href="msg00182.html">[ale] Looking for duplicate photos</a></strong>
</li>
<li>Next by Date:
<strong><a href="msg00184.html">[ale] Looking for duplicate photos</a></strong>
</li>
<li>Previous by thread:
<strong><a href="msg00182.html">[ale] Looking for duplicate photos</a></strong>
</li>
<li>Next by thread:
<strong><a href="msg00185.html">[ale] Looking for duplicate photos</a></strong>
</li>
<li>Index(es):
<ul>
<li><a href="maillist.html#00183"><strong>Date</strong></a></li>
<li><a href="threads.html#00183"><strong>Thread</strong></a></li>
</ul>
</li>
</ul>

<!--X-BotPNI-End-->
<!--X-User-Footer-->
<!--X-User-Footer-End-->
</body>
</html>