Terse Systems

Writing an SBT Plugin

| Comments

One of the things I like about SBT is that it’s interactive. SBT stays up as a long running process, and you interact with it many times, while it manages your project and compiles code for you.

Because SBT is interactive and runs on the JVM, you can use it for more than just builds. You can use it for communication. Specifically, you can use it to make HTTP requests out to things you’re interested in communicating with.

Unfortunately, I knew very little about SBT plugins. So, I talked to Christopher Hunt and Josh Suereth, downloaded eigengo’s sbt-mdrw project, read the activator blog post on markdown and then worked it out on the plane back from Germany.

I made a 0.13 SBT plugin that uses the ROME RSS library to display titles from a list of RSS feeds. It’s available from https://github.com/wsargent/sbt-rss and has lots of comments.

The SBT RSS plugin adds a single command to SBT. You type rss at the console, and it displays the feed:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
> rss
[info] Showing http://typesafe.com/blog/rss.xml
[info]      Title = The Typesafe Blog
[info]      Published = null
[info]      Most recent entry = Scala Days Presentation Roundup
[info]      Entry updated = null
[info] Showing http://letitcrash.com/rss
[info]      Title = Let it crash
[info]      Published = null
[info]      Most recent entry = Reactive Queue with Akka Reactive Streams
[info]      Entry updated = null
[info] Showing https://github.com/akka/akka.github.com/commits/master/news/_posts.atom
[info]      Title = Recent Commits to akka.github.com:master
[info]      Published = Thu May 22 05:51:21 EDT 2014
[info]      Most recent entry = Fix fixed issue list.
[info]      Entry updated = Thu May 22 05:51:21 EDT 2014

Let’s show how it does that.

First, the build file. This looks like a normal build.sbt file, except that there’s a sbtPlugin setting in it:

build.sbtlink
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
// this bit is important
sbtPlugin := true

organization := "com.typesafe.sbt"

name := "sbt-rss"

version := "1.0.0-SNAPSHOT"

scalaVersion := "2.10.4"

scalacOptions ++= Seq("-deprecation", "-feature")

resolvers += Resolver.sonatypeRepo("snapshots")

libraryDependencies ++= Seq(
  // RSS fetcher (note: the website is horribly outdated)
  "com.rometools" % "rome-fetcher" % "1.5.0"
)

publishMavenStyle := false

/** Console */
initialCommands in console := "import com.typesafe.sbt.rss._"

Next, there’s the Plugin scala code itself.

SbtRss.scalalink
1
2
3
object SbtRss extends AutoPlugin {
   // stuff
}

So, the first thing to note is the AutoPlugin class. The Plugins page talks about AutoPlugin — all you really need to know is if you define an autoImport object with your setting keys and then import it into an AutoPlugin, you will make the settingKey available to SBT.

The next bit is the globalSettings entry:

SbtRss.scalalink
1
2
3
override def globalSettings: Seq[Setting[_]] = super.globalSettings ++ Seq(
  Keys.commands += rssCommand
)

Here, we’re saying we’re going to add a command to SBT’s global settings, by merging it with super.globalSettings.

The next two bits detail how to create the RSS command in SBT style.

SbtRss.scalalink
1
2
3
4
5
/** Allows the RSS command to take string arguments. */
private val args = (Space ~> StringBasic).*

/** The RSS command, mapped into sbt as "rss [args]" */
private lazy val rssCommand = Command("rss")(_ => args)(doRssCommand)

Finally, there’s the command itself.

SbtRss.scalalink
1
2
3
4
5
def doRssCommand(state: State, args: Seq[String]): State = {
  // do stuff

  state
}

The first thing we need to do within a command is call Project.extract(state). This gives us a bunch of useful settings such as currentRef, which we can use to pull the value of the SettingKey out. The SBT documentation on Build State – Project related data shows some more examples:

SbtRss.scalalink
1
2
3
4
5
// Doing Project.extract(state) and then importing it gives us currentRef.
// Using currentRef allows us to get at the values of SettingKey.
// http://www.scala-sbt.org/release/docs/Build-State.html#Project-related+data
val extracted = Project.extract(state)
import extracted._

Once we have the extracted.currentRef object, we can pull out the list of URLs with this construct, where the documentation is from Build State – Project data:

SbtRss.scalalink
1
val currentList = (rssList in currentRef get structure.data).get

And then we can put that together with the ROME library to print something out.

SbtRss.scalalink
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
package com.typesafe.sbt.rss

import sbt._
import Keys._
import sbt.complete.Parsers._

import java.net.URL
import com.rometools.fetcher._
import com.rometools.fetcher.impl._
import com.rometools.rome.feed.synd._

import scala.util.control.NonFatal

/**
 * An autoplugin that displays an RSS feed.
 */
object SbtRss extends AutoPlugin {

  /**
   * Sets up the autoimports of setting keys.
   */
  object autoImport {
    /**
     * Defines "rssList" as the setting key that we want the user to fill out.
     */
    val rssList = settingKey[Seq[String]]("The list of RSS urls to update.")
  }

  // I don't know why we do this.
  import autoImport._

  /**
   * An internal cache to avoid hitting RSS feeds repeatedly.
   */
  private val feedInfoCache = HashMapFeedInfoCache.getInstance()

  /**
   * An RSS fetcher, backed by the cache.
   */
  private val fetcher = new HttpURLFeedFetcher(feedInfoCache)

  /** Allows the RSS command to take string arguments. */
  private val args = (Space ~> StringBasic).*

  /** The RSS command, mapped into sbt as "rss [args]" */
  private lazy val rssCommand = Command("rss")(_ => args)(doRssCommand)

  /**
   * Adds the rssCommand to the list of global commands in SBT.
   */
  override def globalSettings: Seq[Setting[_]] = super.globalSettings ++ Seq(
    Keys.commands += rssCommand
  )

  /**
   * The actual RSS command.
   *
   * @param state the state of the RSS application.
   * @param args the string arguments provided to "rss"
   * @return the unchanged state.
   */
  def doRssCommand(state: State, args: Seq[String]): State = {
    state.log.debug(s"args = $args")

    // Doing Project.extract(state) and then importing it gives us currentRef.
    // Using currentRef allows us to get at the values of SettingKey.
    // http://www.scala-sbt.org/release/docs/Build-State.html#Project-related+data
    val extracted = Project.extract(state)
    import extracted._

    // Create a new fetcher event listener attached to the state -- this gives
    // us a way to log the fetcher events.
    val listener = new FetcherEventListenerImpl(state)
    fetcher.addFetcherEventListener(listener)

    try {
      if (args.isEmpty) {
        // This is the way we get the setting from rssList := Seq("http://foo.com/rss")
        // http://www.scala-sbt.org/release/docs/Build-State.html#Project+data
        val currentList = (rssList in currentRef get structure.data).get
        for (currentUrl <- currentList) {
          val feedUrl = new URL(currentUrl)
          printFeed(feedUrl, state)
        }
      } else {
        for (currentUrl <- args) {
          val feedUrl = new URL(currentUrl)
          printFeed(feedUrl, state)
        }
      }
    } catch {
      case NonFatal(e) =>
        state.log.error(s"Error ${e.getMessage}")
    } finally {
      // Remove the listener so we don't have a memory leak.
      fetcher.removeFetcherEventListener(listener)
    }

    state
  }

  def printFeed(feedUrl:URL, state:State) = {
    // Allows us to do "asScala" conversion from java.util collections.
    import scala.collection.JavaConverters._

    // This is a blocking operation, but we're in SBT, so we don't care.
    val feed = fetcher.retrieveFeed(feedUrl)
    val title = feed.getTitle.trim()
    val publishDate = feed.getPublishedDate
    val entries = feed.getEntries.asScala
    val firstEntry = entries.head

    // The only way to provide the RSS feeds as a resource seems to be to
    // have another plugin extend this one.  The code's small enough that it
    // doesn't seem worth it.
    state.log.info(s"Showing $feedUrl")
    state.log.info(s"\t\tTitle = $title")
    state.log.info(s"\t\tPublished = $publishDate")
    state.log.info(s"\t\tMost recent entry = ${firstEntry.getTitle.trim()}")
    state.log.info(s"\t\tEntry updated = " + firstEntry.getUpdatedDate)
  }

  /**
   * Listens for RSS events.
   *
   * @param state
   */
  class FetcherEventListenerImpl(state:State) extends FetcherListener {
    def fetcherEvent(event:FetcherEvent) = {
      import FetcherEvent._
      event.getEventType match {
        case EVENT_TYPE_FEED_POLLED =>
          state.log.debug("\tEVENT: Feed Polled. URL = " + event.getUrlString)
        case EVENT_TYPE_FEED_RETRIEVED =>
          state.log.debug("\tEVENT: Feed Retrieved. URL = " + event.getUrlString)
        case EVENT_TYPE_FEED_UNCHANGED =>
          state.log.debug("\tEVENT: Feed Unchanged. URL = " + event.getUrlString)
      }
    }
  }
}

This is an intentionally trivial example, but it’s easy to show how you could use this to check if the build failed, for example. Have fun.

Comments