Category Archives: Work

Xcode Server Slack Integration

After spending a day last week struggling to get a working integration of Xcode Server and Slack (the popular messaging platform) I finally have arrived upon a reliable solution. What started out as a long bash script (and sed, and cut, and tr) that eventually called a Python program, is now just a Python program run by the Xcode ‘Trigger Script’ functionality. Read on for how I did it.

Working with a client on an iOS app we wanted to keep everyone aware of the project’s progress and the current state of all tests, both server and iOS. So, our team added the client as a Slack single-channel guest and got to work integrating our server-side Strider Continuous Integration server and Xcode Server.

Not knowing any better I dove into the output of the Xcode Build located in /Library/Developer/XcodeServer/IntegrationAssets and poked around. I found the file buildService.log which contained build output detailing everything I needed to know about the build, looking like this:

[BuildService] Build results summary: {
    analyzerWarningChange = 0;
    analyzerWarningCount = 0;
    errorChange = 2;
    errorCount = 2;
    improvedPerfTestCount = 0;
    regressedPerfTestCount = 0;
    testFailureChange = 0;
    testFailureCount = 0;
    testsChange = 3;
    testsCount = 3;
    warningChange = 0;
    warningCount = 0;
}

[BuildService] Build results summary: {

analyzerWarningChange = 0;

analyzerWarningCount = 0;

errorChange = 2;

errorCount = 2;

improvedPerfTestCount = 0;

regressedPerfTestCount = 0;

testFailureChange = 0;

testFailureCount = 0;

testsChange = 3;

testsCount = 3;

warningChange = 0;

warningCount = 0;

}

Which, I immediately thought, “I know, I’ll tackle it with bash and sed!”. So I got to work writing a script to parse these entries and ultimately distribute them out to our Slack channel so everyone can see progress as we go along. Here’s the bash script I wrote:

#!/bin/bash

# Build the basic file paths and variables to be used
LINES_IN_SUMMARY=12
XCODE_BUILD_DIR="/Library/Developer/XcodeServer/IntegrationAssets"
BUILD_SERVICE_LOG=buildService.log
# Currently we're only using 1 bot. If this changes the following will need to change
BOT_DIR="$XCODE_BUILD_DIR"/$(ls "$XCODE_BUILD_DIR" | tail -1)
LAST_BUILD_DIR="$BOT_DIR"/$(ls -Art "$BOT_DIR" | tail -n1)
RESULTS_FILE="$LAST_BUILD_DIR"/"$BUILD_SERVICE_LOG"

echo "Evaluating file $RESULTS_FILE"

# Get the line number of the beginning of the summary block
LINE_NUMBER=$(grep -n "Build results summary" "$RESULTS_FILE" | cut -f1 -d:)
# Now, get the line number of the end of the block
END_OF_LOG=$((LINE_NUMBER + LINES_IN_SUMMARY))

# This prints from the found regexp to the end of the block summary
BUILD_RESULT_VARIABLES=$(sed -n -e "/Build results summary/,$END_OF_LOG p" "$RESULTS_FILE")

# Loop through the results list which looks like:
# testsCount = 2;
# errorCount = 0;
# We split each line out looking for specific text then store it in a variable
while read -r line; do
echo "$line"
if [[ "$line" == *"testsCount"* ]]
then
TESTS_COUNT=$(cut -d "=" -f2 &lt;&lt;&lt; "$line" | tr -d ' ' | tr -d ';')
fi
if [[ "$line" == *"errorCount"* ]]
then
ERROR_COUNT=$(cut -d "=" -f2 &lt;&lt;&lt; "$line" | tr -d ' ' | tr -d ';')
fi
if [[ "$line" == *"warningCount"* ]]
then
WARNING_COUNT=$(cut -d "=" -f2 &lt;&lt;&lt; "$line" | tr -d ' ' | tr -d ';')
fi
done &lt;&lt;&lt; "$BUILD_RESULT_VARIABLES"

python -c "import argparse; import slack;  import slack.chat; slack.api_token = 'slack_token';  slack.chat.post_message('#slack_channel', 'Xcode test completed ' + \"$TESTS_COUNT\" + ' tests with ' + \"$ERROR_COUNT\" + ' errors, and ' + \"$WARNING_COUNT\" + ' warnings.', username='XcodeBot')"

echo "Tests count: $TESTS_COUNT"
echo "Error count: $ERROR_COUNT"
echo "Warning count: $WARNING_COUNT"

#!/bin/bash

# Build the basic file paths and variables to be used

LINES_IN_SUMMARY=12

XCODE_BUILD_DIR="/Library/Developer/XcodeServer/IntegrationAssets"

BUILD_SERVICE_LOG=buildService.log

# Currently we're only using 1 bot. If this changes the following will need to change

BOT_DIR="$XCODE_BUILD_DIR"/$(ls "$XCODE_BUILD_DIR" | tail -1)

LAST_BUILD_DIR="$BOT_DIR"/$(ls -Art "$BOT_DIR" | tail -n1)

RESULTS_FILE="$LAST_BUILD_DIR"/"$BUILD_SERVICE_LOG"

echo "Evaluating file $RESULTS_FILE"

# Get the line number of the beginning of the summary block

LINE_NUMBER=$(grep -n "Build results summary" "$RESULTS_FILE" | cut -f1 -d:)

# Now, get the line number of the end of the block

END_OF_LOG=$((LINE_NUMBER + LINES_IN_SUMMARY))

# This prints from the found regexp to the end of the block summary

BUILD_RESULT_VARIABLES=$(sed -n -e "/Build results summary/,$END_OF_LOG p" "$RESULTS_FILE")

# Loop through the results list which looks like:

# testsCount = 2;

# errorCount = 0;

# We split each line out looking for specific text then store it in a variable

while read -r line; do

echo "$line"

if [[ "$line" == *"testsCount"* ]]

then

TESTS_COUNT=$(cut -d "=" -f2 <<< "$line" | tr -d ' ' | tr -d ';')

if [[ "$line" == *"errorCount"* ]]

then

ERROR_COUNT=$(cut -d "=" -f2 <<< "$line" | tr -d ' ' | tr -d ';')

if [[ "$line" == *"warningCount"* ]]

then

WARNING_COUNT=$(cut -d "=" -f2 <<< "$line" | tr -d ' ' | tr -d ';')

done <<< "$BUILD_RESULT_VARIABLES"

python -c "import argparse; import slack; import slack.chat; slack.api_token = 'slack_token'; slack.chat.post_message('#slack_channel', 'Xcode test completed ' + \"$TESTS_COUNT\" + ' tests with ' + \"$ERROR_COUNT\" + ' errors, and ' + \"$WARNING_COUNT\" + ' warnings.', username='XcodeBot')"

echo "Tests count: $TESTS_COUNT"

echo "Error count: $ERROR_COUNT"

echo "Warning count: $WARNING_COUNT"

This seemed to work! But, however, proved problematic as I discovered (only after I’d written the script!) that there lies a chicken-and-egg problem with when Xcode will execute a script versus when it’s all done packaging up the buildService.log files. The pattern that Xcode follows is: run the project tests, run the post-process script, then build the buildService.log files. So, the script above was running on a previous build, not the most current one.

I poked around a bit more but ultimately couldn’t come up with a solution of how to solve this. So, I searched around on Twitter and found @mjmoriarity, a wonderfully helpful Apple engineer who pointed me towards my ultimate solution of using Xcode’s set environment variables. Using these, my script was now reduced to a handful of lines and is wonderfully reliable:

env

# The Xcode build system sets environment variables which we’re using in our Python script
echo “Total: $XCS_TESTS_COUNT”
echo “Errors: $XCS_ERROR_COUNT”
echo “Warnings: $XCS_WARNING_COUNT”

python -c "import slack; import slack.chat; slack.api_token = 'slack_token'; slack.chat.post_message('#slack_channel', \"$XCS_BOT_NAME\" + ' test #' + \"$XCS_INTEGRATION_NUMBER\" + ' completed with the result: ' + \"$XCS_INTEGRATION_RESULT\" + '. There were ' + \"$XCS_TESTS_COUNT\" + ' tests with ' + \"$XCS_ERROR_COUNT\" + ' errors, and ' + \"$XCS_WARNING_COUNT\" + ' warnings.', username='XcodeBot')"

env

# The Xcode build system sets environment variables which we’re using in our Python script

echo “Total: $XCS_TESTS_COUNT”

echo “Errors: $XCS_ERROR_COUNT”

echo “Warnings: $XCS_WARNING_COUNT”

python -c "import slack; import slack.chat; slack.api_token = 'slack_token'; slack.chat.post_message('#slack_channel', \"$XCS_BOT_NAME\" + ' test #' + \"$XCS_INTEGRATION_NUMBER\" + ' completed with the result: ' + \"$XCS_INTEGRATION_RESULT\" + '. There were ' + \"$XCS_TESTS_COUNT\" + ' tests with ' + \"$XCS_ERROR_COUNT\" + ' errors, and ' + \"$XCS_WARNING_COUNT\" + ' warnings.', username='XcodeBot')"

Peripheral Intimidation

Driving from Seattle to Portland with some regularity gives me a chance to stop (and go, and stop) and think for a bit. As with every car drive I begin to mull over what’s been on my mind recently – usually development-related issues. The last drive I stumbled across a comparison that helps explain a distraction I’d been experiencing recently in my work. The analogy? Software and semi’s.

Being a part of any development community means being exposed every day to amazing work done by amazing people. Continuously seeing all the cool things people are doing it’s hard not to draw comparisons to your own work. The feelings of inadequacy and Impostor Syndrome, unless carefully monitored and controlled, are inevitable and intense. Because the names of the people creating these apps all seem to blend together after a while, it feels like one single, impressive, superhuman coding all these apps, sites, and documenting every step of the way. Someone more talented, thoughtful, creative, and with more perseverance than you could ever dream of. In essence, a hulking semi of app-creating power rocketing down the information superhighway (see what I did there).

It’s natural to feel small compared to these semi’s. In fact, it’s probably healthy as it pushes us to continue to sharpen our saws and elevate our skills.

But it’s so critical to remember that each cool thing being churned out of this great internet machine was created by regular people. People who found a niche, pounded hard at it, and after many failures and false starts arrived at something close to the final product.

Mike Lee, a developer I’ve long respected for his talent and design skills, was recently interviewed and outed that he’s a just regular guy, not superhuman. If you haven’t yet seen it (or listened to the MP3) it’s well worth your time. One of the smartest guys running in Mac/iOS development circles is an ordinary dude. It doesn’t get much more inspiring than that.

So, just as you continue to suck it up and drive down freeways past the hulking semis, development should continue as well.

Access Headless Local Machine With Dropbox

samswiches

sandwiches by sam

Category Archives: Work

Xcode Server Slack Integration

Peripheral Intimidation

Access Headless Local Machine With Dropbox

First Days