Tales of an IT Nobody

devbox:~$ iptables -A OUTPUT -j DROP

Grepping extremely large files November 28, 2011

So you forgot to set up logrotate on an active log eh? You’ve got a many gigabyte file to weed through and you need to extract a chunk of time from it?

Here’s a quick cheat sheet to help you get by, quickly and sanely.

It’s about byte offsets!

  • Get the byte offset in the file where your time range starts
  • Get the byte offset in the file where your time range ends
  • dd the data out!

Caveats

  • You should tack on extra bytes to the byte length, because the offset_end number is actually the beginning byte of your boundary log entry
  • Figuring out the boundary is a bit tricky because a log entry -has- to be present in order to match, so if you’re looking for what happened at 20:00 hours on X date, you may have to round up to the date level depending on how busy your log is
  • This is just a trick to extract a chunk of entries to speed up further filtering.

Full example

No Comments on Grepping extremely large files
Categories: linux servers

Worthy of distribution: Cloud analogy November 5, 2011

This post on Beyond Bandwidth seems to summarize some of my feelings about cloud computing – it’s best thought of as an outsourcing task for the most part; Although the benefits of something like an extra DNS server are a bit more than an ‘outsource benefit’; but you get the idea:

Cloudy analogies with a chance of illusion

No Comments on Worthy of distribution: Cloud analogy
Categories: security servers