It’s pretty simple. In your config/environment.rb:
config.gem 'activerecord-sqlserver-adapter', :source => 'http://gems.rubyonrails.org', :lib => 'active_record/connection_adapters/sqlserver_adapter'
Then run rake gems:install and away you go.
Hugh has some more tips about the packages and configuration required to connect to an SQL Server from a Linux platform.