Category Archives: technology

robocopy-fu

Here’s a little powershell script I used to solve a problem where I needed to synchronize MILLIONS of files quickly and be notified about it promptly once the operation has complete.

Normally, I’d take machine snapshots or something but this backup is native and sadly, it happens at the file level; it also requires a Microsoft OS to run natively on bare-metal. rsync isn’t fast enough as-tested. However, 128 threads of robocopy flies right through it!

Robocopy starting with Windows Vista *SHOULD* support multiple concurrent threads with the /MT switch.

# PowerShell Robocopy script with e-mail notification
# Created by Michel Stevelmans - http://www.michelstevelmans.com
# needed moar variables and threads [http://jcph.am edit]

# Change these values, name your job, configure the paths
$job = "BLAH"
$DateAndTime = get-date -UFormat "%Y%m%d_%H%M%S"
$SourceFolder = "F:\REPORTS"
$DestinationFolder = "\\UNC_PATH\TO\SHARE\REPORTS"
$Logfile = "C:\LogFiles\$job-$DateAndTime.log"
$EmailFrom = "some@server.com"
$EmailTo = "deez@deez.com"
$EmailBody = "Robocopy Job: $job completed successfully. See attached log: $Logfile for details"
$EmailSubject = "Robocopy Summary: $job for $DateAndTime"
$SMTPServer = "some.smtp.server"
$SMTPPort = "25"

# Copy Folder with Robocopy
Robocopy $SourceFolder $DestinationFolder /R:1 /W:1 /MT:128 /MIR /XX /LOG:$Logfile /NP /nfl /ndl

# Send E-mail message with log file attachment
$Message = New-Object Net.Mail.MailMessage($EmailFrom, $EmailTo, $EmailSubject, $EmailBody)
$Attachment = New-Object Net.Mail.Attachment($Logfile, 'text/plain')
$Message.Attachments.Add($Attachment)
$SMTPClient = New-Object Net.Mail.SmtpClient($SmtpServer, $SMTPPort)
$SMTPClient.Send($Message)
Advertisements

problem solving: barrel locks

I often refer to myself as a problem solver by nature. I work on computers, manage networks, engineer projects, and design and build things at varying scales, but where I feel most rewarded is in the moment of solving the problem. That stressful instance where I am presented with some new challenge and must conquer it.

The very moment when I can say “Ah-ha!” in my head, and no one else has seen the solution yet, that’s the moment I want to live in forever.

Problem:

Failed hard drive in a 3 drive RAID5 array. Production email server. No key can be sourced that fits the lock. Need to get the drive out to replace it. Customer claims to me “it has been that way for months”. Meanwhile, I’m shaking my head and explaining the limitations of RAID5 redundancy. So we locate a lot of barrel keys that do not work and none that ultimately unlock the drive from its caddy.

I want to replace the drive during business hours – while I am at work – instead of after hours. I do not want to destroy the lock or risk damage to the server. The best suggestions I have are to drill out the lock or contact the manufacturer and attempt to locate a replacement.

key fail

Solution:

File down slightly larger key with hand file until it fits the lock. Locks are meant to be defeated by keys. Without a key, what purpose does a lock have?

filed key filed key filed key

Like a boss, it works on the first try.

No joke. I came home last night with a task in my head like this, “I’m going to file down this key and get it to work tomorrow.”

And today, I did exactly that:

success!

Visualize the solution and make it reality.

skynet-tor-powered-botnet

I remember reading about this on reddit awhile back and thought “wow”.

Now it appears Rapid7 has a full breakdown of this botnet, which you can read about here.

The summary:

“Skynet runs all its C&C servers as Hidden Services and all compromised computers are configured to be part of the Tor network as well.

The advantages of this approach are:

  • The botnet traffic is encrypted, which helps prevent detection by network monitors.
  • By running as an Hidden Service, the origin, location, and nature of the C&C are concealed and therefore not exposed to possible takedowns. In addition, since Hidden Services do not rely on public-facing IP addresses, they can be hosted behind firewalls or NAT-enabled devices such as home computers.
  • Hidden Services provide a Tor-specific .onion pseudo top-level domain, which is not exposed to possible sinkholing.
  • The operator can easily move around the C&C servers just by re-using the generated private key for the Hidden Service.

Long story short, Tor, due to its design and internal mechanics, makes it a perfect protocol for botnets. Because of this, all critical communications of Skynet to its C&C servers are tunneled through a Tor SOCKS proxy running locally on compromised computers.”

Yikes.

 

botnet-op

Meet the Obama campaign’s $250 million fundraising platform

The numbers

6 month life span
$250 million dollars, 4,276,463 donations
81,548,259 pageviews, 17,807,917 unique visitors
60% faster time to paint than previous platform
240 a/b tests, 49% increase in donation conversion rate
1,101 frontend deploys
4,000 lines of JavaScript
23% of traffic is mobile (including tablet)

The web stack

  • Static HTML pages generated by Jekyll
  • Served by Akamai CDN, origin hosted on Amazon S3
  • Version control with GitHub
  • Conversion optimization with Optimizely
  • LESS for CSS preprocessor, JavaScript compiled by CodeKit
  • Redundant, REST payment processor

    read more here

[ hamcomputers ]

Technical Support enthusiasts.

Happenin's in the 'Ham

We are here to keep you updated with all the free, cheap, and amazing events around Birmingham!

CaliBama Belle

A Southern Belle with a Golden State of Mind

We Share the Same Sky

Desiring but another day to ebb the pull, damn the flow and spend an evening wading creeks that meander.

Shedrick Flowers Photography

'dem sexy internets

The Immortal Jukebox

A Blog about Music and Popular Culture

The Bitcoin Wife

All things fresh and fabulous in the Bitcoin world.

Unlearning Economics

Musings on the Current State of Economics

Intellectual Detox

Reconstructing a more accurate view of reality

Thought Catalog

Thought Catalog is a digital youth culture magazine dedicated to your stories and ideas.

AudenX

'dem sexy internets

Bourbon & Kale

A Left Coast Guide to Longevity and Southern Guide to Sanity

EARMILK

'dem sexy internets

Jesse's Café Américain

'dem sexy internets

MineForeman

Bitcoin Stuff

Bitcoin Foundation

'dem sexy internets

%d bloggers like this: