Monday, February 27, 2012

Dart MUD... Progress 1

So before I get started, I just want to put out there that the latest integration build of DartEditor has been released, and you should pick it up here. This build contains the latest IDE, the SDK and now Dartium as well! For those of you running 64-bit systems, it is a 32-bit build of Chromium so you may encounter some library errors if you have not installed the extra components you need.

So now that that news is out of the way.. a little progress report on my Dart MUD. (I may need to come up with a better development name, as it conflicts with the already existing MUD: Dartmud.com) So far I have a minimal server, from which I've managed to remove most unnecessary code from and make the mudlib independent of the server. I have a generic MudLib object which handles most interactions between server and specific mudlib components. A User object which handles the client connections and various user properties (username, passwords, etc). However I'm not completely happy having the socket implementation directly part of the User object, and may break down into a connection object which the User class will either have it as a property, or may inherit from (just because someone connects, doesn't mean it will result in a User on the system).

After some playing around with a static class which contained the various commands a player can execute, I decided to instead create a small class, and each command is an instance of the class. All of which are managed by a CommandDaemon/Command Manager. I can see this allowing for more flexibility in adding commands globally and perhaps even in just specific rooms or objects. On the note of Rooms and GameObjects, I have classes for these also implemented, as well as a 'container' interface and a factory class for it. Users and Rooms both currently extend the factory class for the container.

At this point, I've added a few 'commands', including the exit and shutdown commands from the EchoServer (though now setup to be used with the Command Manager I have implemented). I've also added a basic 'look', 'broadcast' and 'help' command, and am in the process of fleshing out these respective commands so they properly handle any arguments sent to them (broadcast works to notify all users, regardless of if they are in the same room as the person issuing the commands.

I still have so much to do. Currently when connecting you are prompted for a username/password to log in. However you can enter anything an be granted access with the username you enter. I need to be able to create a new user, to save the user, and to load them when logging in. Also I have a very basic room, but only the one. I still have to create more rooms and implement travel commands to move between rooms. To say nothing of the other normal things you would expect on a mud including equipment, objects, money, combat, etc etc etc.

While I still have a very long way to go to make a basic mud, I'm quite pleased with the progress I have been making towards that end. Something I may want to look towards as well is some type of 'reload' or 'refresh' command, as I have to shutdown the server and restart it after each change I make. I'll also need to work on a memory management daemon of some kind too, so that once the mud gets to a decent size, objects will only be loaded/created when accessed, and after a period of time, removed if not used or near an active player.

One of these days I'm going to make an effort to learn git/git hub too.

Thursday, February 23, 2012

Featureless Sockets

Okay, so the title is a bit of a stretch. Dart Sockets are far from featureless. As some of you may or may not know, I've decided to take a segue in my EchoServer towards working on a basic MUD and MUDLib written in Dart. It will be far from full featured, and far from fully functional but it will be the basis and a nice little start.

Within a few minutes of converting the EchoServer to be the basis of the MUD driver, I've already run into an issue of a missing feature with Sockets in Dart. I'm unable to get the host of an incoming connection. That is, I can't find out who is connecting to me. While it's a trivial matter for a MUD, or many other services, one would expect such functionality for various uses including HTTP server logging, etc.

After digging through the bleeding edge to see if it may have been recently included but just not yet updated in the API, I still can't find any reference to such properties. So as such, I've created a bug report over in the Dart issue tracker. If you're interested, or want to star the report you can view it here at: Issue 1819

As I progress with the Dart core and MUDLib I'm sure I'll find some additional bugs to report, or just missing features I can think could be added. Additionally I'll keep progress reports available on here and eventually release the source on GitHub or something similar.

Tuesday, February 21, 2012

Serving Sockets... The Salad

I haven't had the opportunity to do much with Dart lately, due to scheduling. However this afternoon I had a little chance to write some more. I decided that I wanted to modify the EchoServer to maintain the connection and close the connection only when receiving a specific command. I also wanted to add a command to shutdown the server itself completely. In order to properly accommodate this, I needed to refactor the server code a little. I wrapped the ServerSocket in a manager class:

#import('dart:io');

class ServerManager {
  ServerSocket _listenServer;
  
  ServerManager() {
    _listenServer = new ServerSocket("127.0.0.1", 5700, 0);
    
    _listenServer.connectionHandler = this._handleConn;
  }

  void _handleConn(Socket conn) {
    StringInputStream clientIn = new StringInputStream(conn.inputStream);
    
    clientIn.lineHandler = () {
      String input = clientIn.readLine();
      print("Received: $input");
      if(input.toLowerCase() == 'stop') {
        String cls = "** Stopping Server. Closing connection **\n";
        conn.writeList(cls.charCodes(), 0, cls.length);
        conn.close();
        _listenServer.close();
        print("** Stopping Server! **");
      } else if(input.toLowerCase() == 'exit') {
        String cls = "** Closing connection to client **\n";
        conn.writeList(cls.charCodes(), 0, cls.length);
        conn.close();
        print("** Closing connection to client. **");
      } else {
        String output = "${input.toUpperCase()}\n";
        conn.writeList(output.charCodes(), 0, output.length);
        print("Sent: $output");
      }
    };
    
  }
}

void main() {
  ServerManager sMan = new ServerManager();
 
}

So as you can see I made a few changes from my original EchoServer. As mentioned above I wrapped the server in a manager class, this enables me to easily close the server socket without using a global variable. In addition I added a couple clauses to check for the 'stop' or 'exit' commands which will stop the server or just close the client connection respectively. And finally I stopped pulling the output stream of the sockets directly, and instead use the writeList methods directly on the socket itself. I wasn't gaining any real benefit by creating an additional variable for the socket's OutputStream, so I just dropped it altogether.

Now as is, the above will run and accept connections and echo any new lines until the stop or exit commands are received. If the exit command is received, then the server will close the connection to that socket. If stop is received it will close the connection to that socket and then tell the server to close. However because of the event driven nature of the server, the Sockets and ServerSockets are not blocking. That is, even without adding any additional threads (Isolates), we can accept connections from multiple sources. If you open up multiple telnet connections to the host, you can see how you can send data and receive responses on each connection independent of the other.

But this also leaves us with a small issue. If we tell the server to stop from one telnet session while the other is still active.. the server will accept the stop command, and it will schedule the ServerSocket to be stopped, but not until the other socket has been closed. Try it out and you will see that the connection in which we issue the stop command is disconnected, and the console will indicate that the server is stopping. But the other telnet session will remain active until we issue an exit or stop command. Only once the 2nd session is closed will the server stop. And if for some reason the other session does not terminate properly (for instance connection drops or the telnet application is closed before issuing an exit/stop command) then the server will hang, not accepting new connections but not terminating either (assuming eventually the socket will time out but potentially not since I do not have those error handlers in place either).

This may be the desired situation with some servers to shut them down gracefully for instance, however in our EchoServer we want it to shut down immediately if it receives the stop command. So we'll need to keep a list of active connections and iterate through them and close each one, then stop the server. So I ended up with the following:

#import('dart:io');

class ServerManager {
  ServerSocket _listenServer;
  List _socketList;
  
  ServerManager() {
    _socketList = new List();
    _listenServer = new ServerSocket("127.0.0.1", 5700, 0);
    
    _listenServer.connectionHandler = this._handleConn;
  }
  
  void sendStops() {
    List cls = "** Server received stop request. Closing connection to client **\n".charCodes();
    
    while(!_socketList.isEmpty()) {
      Socket conn = _socketList.removeLast();
      conn.writeList(cls, 0, cls.length);
      conn.close();
    }
  }
  
  void _handleConn(Socket conn) {
    _socketList.add(conn);
    StringInputStream clientIn = new StringInputStream(conn.inputStream);
    
    clientIn.lineHandler = () {
      String input = clientIn.readLine();
      print("Received: $input");
      
      if(input.toLowerCase() == 'stop') {
        sendStops();
        _listenServer.close();
        print("** Stopping Server! **");
      } else if(input.toLowerCase() == 'exit') {
        String cls = "** Closing connection to client **\n";
        conn.writeList(cls.charCodes(), 0, cls.length);
        int sockInd = _socketList.indexOf(conn);
        _socketList.removeRange(sockInd, 1);
        conn.close();
        print("** Closing connection to client: $sockInd **");
      } else {
        String output = "${input.toUpperCase()}\n";
        conn.writeList(output.charCodes(), 0, output.length);
        print("Sent: $output");
      }
    };
    
  }
}

void main() {
  ServerManager sMan = new ServerManager();
 
}

As you can see I also added a method sendStops just to iterate through all the sockets, popping them out of list and sending them the stop notice and disconnecting them. I made this separate from the actual stopping of the server in case it should ever be required for any other reason as well. Initially I tried using a Set to hold just unique connections, and provide easier way of removing elements however I found out that there's an issue with Set's in that any values stored in a set must implement Hashable. This wasn't added in the API documentation and it was only after a little digging through the DartBug page and Newsgroup that I found this is 'expected' behaviour. As such, I had to use the list. For a specific 'exit' command I have to get the index of the value and remove it from the list with removeRange with a size of 1 element. I also setup the broadcast message directly to a List of Int's immediately just to avoid having to convert it multiple times as I iterate through the connections. While still missing any error handling, etc. I'm rather pleased with how the server is progressing and in some ways it conjures up images of the old school MUD's. Maybe a project to play with?

Thursday, February 16, 2012

Serving Sockets... The Soup Crackers

So as I mentioned in my previous post, there is more than one way to communicate through sockets. Below I have show 3 different, though similar ways to make use of sockets. This is far from exhaustive and please note that this is for demonstration purposes only and as is, should not be used in production code of any kind. They're missing error checking, stream verification, etc.

I am interested if any one has any comments or feedback as to use cases where they would choose to use one of the particular methods over another. Everything below is fairly procedural and could probably be implemented much cleaner as Objects, but as mentioned this is purely a test case using the EchoServer I wrote a couple posts back.

#import("dart:io");

void main() {
  sockets_with_handlers();
  
  sockets_with_direct_streams();

  sockets_with_socket_streams();
}

void sockets_with_handlers() {
  Socket usingHandlers = new Socket("127.0.0.1", 5700);
  String test = "SocketHandlers Test String\n";
  
  // Called when we successfully connect
  usingHandlers.connectHandler = () {
    print("Handers: Connected");
  };
  
  // Called when we can write to the socket.
  usingHandlers.writeHandler = () {
    usingHandlers.writeList(test.charCodes(), 0, test.length);
    print("Handers Sent: $test");
  };
  
  // Called when we receive from the socket.
  usingHandlers.dataHandler = () {
    int availBytes = usingHandlers.available();
    List buff = new List(availBytes);
    usingHandlers.readList(buff, 0, availBytes);
    print("Handers Received: ${new String.fromCharCodes(buff)}");
  };
  
  // Called when the input stream from socket is closed.
  usingHandlers.closeHandler = () {
    print("Handers: End of stream. Closing connection");
    usingHandlers.close();
  };
}

void sockets_with_direct_streams() {
  Socket usingStreams = new Socket("127.0.0.1", 5700);
  String test = "Direct Socket Streams test string\n";
  
  // To be 'pure' without the handlers we assume connection is successful.
  // NOTE: Not recommended!
  print("Direct Streams: Connected");
  // Wrap the input stream in StringInputStream so we can
  // make use of the convenience functions.
  StringInputStream inStream = new StringInputStream(usingStreams.inputStream);
  OutputStream outStream = usingStreams.outputStream;
  
  // Write to our stream
  outStream.write(test.charCodes());
  print("Direct Streams Sent: $test");
  
  // Use our handle wrapper to read lines
  // Saves us from using dataHandler directly
  inStream.lineHandler = () {
    String input = inStream.readLine();
    print("Direct Streams Received: $input");
  };
  
  // All bytes have been read and input stream is closed.
  inStream.closeHandler = () {
    print("Direct Streams: End of stream. Closing connection");
    usingStreams.close();
  };
}

void sockets_with_socket_streams() {
  Socket usingSocketStreams = new Socket("127.0.0.1", 5700);
  String test = "Socket In/Out Streams test string\n";
  
  // To be 'pure' without the handlers we assume connection is successful.
  // NOTE: Not recommended!
  print("Socket Streams: Connected");
  
  SocketInputStream inStream = new SocketInputStream(usingSocketStreams);
  SocketOutputStream outStream = new SocketOutputStream(usingSocketStreams);
  
  // Write to our stream.
  outStream.write(test.charCodes());
  print("Socket Streams Sent: $test");
  
  // Called when new data arrives in our SocketInputStream
  inStream.dataHandler = () {
    List buff = inStream.read();
    print("Socket Streams received: ${new String.fromCharCodes(buff)}");
  };
  
  // Input stream has been closed. Make sure output stream
  // and socket itself are also closed.
  inStream.closeHandler = () {
    print("Socket Streams: End of stream. Closing connection");
    outStream.close();
    usingSocketStreams.close();
  };
}
Direct Streams: Connected
Direct Streams Sent: Direct Socket Streams test string

Socket Streams: Connected
Socket Streams Sent: Socket In/Out Streams test string

Handers: Connected
Handers Sent: SocketHandlers Test String

Direct Streams Received: DIRECT SOCKET STREAMS TEST STRING
Socket Streams received: SOCKET IN/OUT STREAMS TEST STRING

Handers Received: SOCKETHANDLERS TEST STRING

Direct Streams: End of stream. Closing connection
Socket Streams: End of stream. Closing connection
Handers: End of stream. Closing connection

One of the first things you'll notice with the output is that due to the non-blocking nature of the callbacks, some of the calls happened in near-parallel. In the sockets_with_socket_streams function, while I didn't specifically make use of it, we could have also wrapped the SocketInputStream with a StringInputStream as well, similar to what we did in the sockets_with_direct_streams function.

Serving Sockets... The Soup

Alright, first thing first. Announced during yesterday's Episode 2 of Dartisans Hangout, Dartium binaries are being relased. Initially for Mac OS X and Linux with Windows binaries to follow soon. I won't go on about this much as it has already been mentioned again and again, and again.

One thing to note however, is that currently the Linux Build is a 32-bit version so if you're running a 64-bit OS you will need to ensure you have the proper 32-bit libraries downloaded and installed. Or alternatively, wait for the 64-Bit builds to arrive.

Continuing with my small series on Sockets in the Dart:IO library, I thought well I have an EchoServer written, albeit extremely primitive, my next 'logical' step is to make a client which connects to the server, sends a string, receives the echo response and disconnects. This should be pretty straight forward, since it was so easy to make the server... Oh boy was I ever wrong about that. Not that Sockets are overly complex or anything, they just have many more ways of accomplishing the same thing, but they can't be used in conjunction as I found out with the following:

#import("dart:io");

void main() {
  // Create a new socket connecting to localhost and port 5700
  // the same port as our echo server we wrote is running on.
  Socket conn = new Socket("127.0.0.1", 5700);
  StringInputStream inputStr;
  OutputStream outputStream;
  
  // method is called when connection is established.
  conn.connectHandler = () {
    print("Now Connected");
    inputStr = new StringInputStream(conn.inputStream);
    outputStream = conn.outputStream;
    
    String test = "This is a simple test\n";
    outputStream.write(test.charCodes());
    print("Sending: $test");
    
    // We wrapped the input stream for this easier reading
    // using the lineHandler as opposed to bulk data.
    inputStr.lineHandler = () {
      String input = inputStr.readLine();
      print("Recieved: $input\n");
    };
  };
  
  // Called when the last byte of data has been read from the socket.
  // Socket potentially still open for writing.
  conn.closeHandler = () {
    print("Connection closed. Last byte of data has been read from stream");
    conn.close();
  };
  
}
Now Connected
Unhandled exception:
StreamException: Cannot get input stream when socket handlers are used
 0. Function: '_Socket@14117cc4.get:inputStream' url: 'dart:io' line:4231 col:9
 1. Function: '::function' url: 'src/dart/echo/EchoClient.dart' line:13 col:54
 2. Function: '_Socket@14117cc4.firstWriteHandler' url: 'dart:io' line:4274 col:64
 3. Function: '_SocketBase@14117cc4._multiplex@14117cc4' url: 'dart:io' line:3898 col:23
 4. Function: '_SocketBase@14117cc4.function' url: 'dart:io' line:3996 col:59
 5. Function: 'ReceivePortImpl._handleMessage@924b4b8' url: 'bootstrap_impl' line:1734 col:22

So as we can see, this didn't work quite as cleanly as we'd have liked. Apparently the issue above is with my mixing the Socket handlers, such as connectHandler and closeHandler, while also pulling the socket's input and output streams themselves. In the case of the InputStream wrapping it in a StringInputStream handler to make dealing with input a little easier since we only expect text.

Initially I thought this error was telling me that I cannot use any socket handlers, if I wanted to pull the IO streams directly from the Socket. However I only just now stumbled upon an easier solution. The error isn't with the use of connectHandler and closeHandler on the socket. Apparently only the closeHandler is triggering the error. So I can move that method to be called on the StringInputStream instead. So a quick re-write gives me the following:

#import("dart:io");

void main() {
  // Create a new socket connecting to localhost and port 5700
  // the same port as our echo server we wrote is running on.
  Socket conn = new Socket("127.0.0.1", 5700);
  StringInputStream inputStr;
  OutputStream outputStream;
  
  // method is called when connection is established.
  conn.connectHandler = () {
    print("Now Connected");
    inputStr = new StringInputStream(conn.inputStream);
    outputStream = conn.outputStream;
    
    String test = "This is a simple test\n";
    outputStream.write(test.charCodes());
    print("Sending: $test");
    
    // We wrapped the input stream for this easier reading
    // using the lineHandler as opposed to bulk data.
    inputStr.lineHandler = () {
      String input = inputStr.readLine();
      print("Recieved: $input\n");
    };

    // Called when the last byte of data has been read from the socket.
    // Socket potentially still open for writing.
    inputStr.closeHandler = () {
      print("Connection closed. Last byte of data has been read from stream");
      conn.close();
    };
  };
  
}
Now Connected
Sending: This is a simple test

Recieved: THIS IS A SIMPLE TEST

Connection closed. Last byte of data has been read from stream

Well then, that's working so much better. And in fact I could leave it here.. and will for the time being. However later, perhaps tomorrow, I will write a continuation of this post on some of the other methods of using and communicating with sockets. They are plentiful.

Wednesday, February 15, 2012

Servering Sockets... The Appetizer

So I decided it was time to start playing around with some Sockets. As we already know, I'm not a huge fan of the chat sample that's available, as I find it a little too much. Fortunately the Sockets and ServerSockets are not too different from the Process API, so keeping that in mind I wrote a dead simple echo server. This does not adhere to any protocols or anything of that nature. It just accepts input from a client and sends it back to the client. Truth be told, I should be a little ashamed of the following code as its far from complete, far from safe, and far from extendable. But it was a good starting point and I'll be able to now take it and re-factor it convert it into something usable. The goal was simple, to write a simple server and have evidence that communication was bi-directional.

So first I'll provide the code and then a little discussion about it.

#import('dart:io');

void main() {
  // Create a server and bind it to an address and port number.
  ServerSocket listenServer = new ServerSocket("127.0.0.1", 5700, 0);
  
  // This is called when a connection is received.
  listenServer.connectionHandler = handleConn;
}

// Socket conn is the incoming connection.
void handleConn(Socket conn) {
  // Wrap the input from the connection around a StringInputStream to make reading it easier.
  StringInputStream clientIn = new StringInputStream(conn.inputStream);
  // Get the output stream too so we can write back to the client.
  OutputStream clientOut = conn.outputStream;
  
  // Line handler is called when a new line is received to the StringInputStream
  // which we've wrapped around the InputStream from the socket.
  clientIn.lineHandler = () {
    String input = clientIn.readLine();
    print("Received: $input");
    String output = "${input.toUpperCase()}\n"; 
    clientOut.write(output.charCodes());
    print("Sent: $output");
    conn.close();
  };
  
}

So as you can see above the code is very short and simple. I recommend running this in the IDE as currently the process will run forever. Later on I may add a command to shutdown the server from the client. To test out the above code start the server with the DartVM then open a telnet session. Telnet to 'localhost:5700' Then simply type a line of text and hit enter. You should get the same line of text sent back to you but uppercase.

Basically we create the server, and then just handle any incoming connections with it. The connectionHandler is called when a new connection is established to the server, and the function is passed the Socket to the incoming connection.
From that socket we get the InputStream and OutputStreams. For the InputStream we wrap it in a StringInputStream which simplifies reading strings from the connection, and since we're not really worried about receiving any bulk or binary data as such.

Next, thanks to the StringInputStream, we setup a lineHandler which is called when a new line of text is received by the stream. Then we use readLine to store that line in a variable.

We print what we've received to the console, then create a new string which is uppercase version of the line we just received, and add a new line character to the end of it. We want to add the newLine character because when we use the readLine method, it strips the new line character for us automatically. In this case I wanted to return an uppercase string of the original just for visual confirmation that what is being returned is different in some way from the original so it can't be blamed as a local echo by the telnet client or anything like that.

Next we write the output back to the output stream. Because the OutputStream is not wrapped by a String stream we need to convert our string into a list of character codes which are then in turn sent back through the SocketOutputStream. We also write to the console what we sent.

Finally we close the socket. In a later example we may leave this open until we send a specific comment to terminate the client connection, or specifically have the client close the connection instead.

As I mentioned, this example is pretty much as simple as you can get. It also is missing any error handling, any protocol conformation, but it gave me a comfortable start/introduction to sockets in general. If you have any suggestions or comments regarding the code by all means I eagerly look for feedback or suggestions.

Wednesday, February 8, 2012

String that Process

So a comment from Florian Loitsch on my previous post indicated that as opposed to creating a new string from character codes as I was doing for handling output of the process, I can use the existing StringInputStream to automatically wrap the basic input stream into a string. As indicated by the API I can then pull the strings on a per-line basis or as a string chunk. I've revised my example to put both methods to use. Before commenting on it, lets see the code:
#import("dart:io");

void main() {
  // Create a new process with list of arguments and start it.
  Process proc = new Process.start('/usr/bin/snmpget', ['-v', '2c', '-c', 'public', '127.0.0.1', '.1.3.6.1.2.1.1.1.0']);
  
  // Once started, handle the process including receiving input.
  proc.startHandler = beginHandle() {
    print("Inside of start Handler");
    
    // Create StringInputStreams from the basic input streams of the process.
    StringInputStream res = new StringInputStream(proc.stdout);
    StringInputStream err = new StringInputStream(proc.stderr);
    
    // A handle a new line of text added to the input stream (stdout).
    // readLine does not include the terminating new-line character
    res.lineHandler = showLine() {
      print("Inside InputStringStream lineHandler");
      print(res.readLine());
    };
    
    // Handle new data sent to the input stream (stderr)
    // This happens whenever any data is available and does not
    // wait for a new line.
    err.dataHandler = errRead() {
      print("Inside InputStringStream dataHandler");
      print(err.read());
    };
  }; // End off Proc.startHandler
  
  // When the process has ended this handler is called
  // passes process's exit code.
  proc.exitHandler = exitHandle(int exitCode) {
    print("Inside of exit Handler");
    print("Exit code is: ${exitCode}");
    proc.close();
  };
  
  // Called when an error occurs when spawning the process.
  proc.errorHandler = errorHandle(ProcessException errorMsg) {
    print("Inside of Error Handler");
    print(errorMsg);
  };
}

I've also added some comments to the above to help clear it up a little as well. With the particular process I'm launching generally I'm only interested in parsing one return line at a time. So as such with my StringInputStream for stdout I decided to use the lineHandler method. Because stderr may produce more than one line of error, I used the dataHandler trigger the reads. Note that the two are mutually exclusive. That is the same StringInputStream can only define a lineHandler or a dataHandler. Not both.

The lineHandler is called each time a new line is available to the stream. dataHandler is called anytime there's new data available, no matter if it contains a new line or not. This is good if you want to receive a bulk of a string, or potentially expect an unterminated string.

Note I also did not methods for the errorHandlers in this simple example, nor verify if the stream is currently open. Using both this method and the previous one of directly using the basic InputStreams are roughly the same size, however unless you explicitly require the raw character codes, I would personally suggest using the StringInputStream over the basic InputStream for potential performance gains and to help future-proof your code against any internal library chances which make affect results or performance down the road with the InputStream and string from character codes.

Tuesday, February 7, 2012

The INs and OUTs of IO.

So, as per my previous post, I've started to play with the dart:io library. While there is a sample of its usage to create a chat server available in the bleeding edge repository, but in all honesty that is far more complete and thorough than anything I plan on just yet. So essentially I'm just going off of the API documentation. Which, while good, is not exactly bursting with usage examples.

As I mentioned in my previous post, currently the goal of the script is to just run an external command and output the results. The command I chose to work with in this case is snmpget for conducting SNMP queries on equipment. This is a precursor to some further scripting I would do with the VM and Dart in general on both administrative fronts and potentially a full web-app as well.

So I focus my attention on the Process API. Now before I go any further I want to call to attention that I am far from an expert in Dart, or even Javascript for that matter. Most of my programming experiences lay in other area such as perl, python, and then Ruby. The particular use of callbacks in the manner they are used is something not entirely familiar with me and I am quite possibly breaking a few rules with them. If you have any notes or suggestions on improving/securing my simple code by all means feel free to leave some feedback in the comments!

So looking at the API I see I obviously need to create a new Process object with Process.start. I will then also need to have a startHandler and we'll throw in a simple errorHandler just incase I mess up something. Looking at the information for close I should also put in an exitHandler. In fact the close information gives me my first hint as to where the process logic should be contained. According to the documents for close, Usually, close should be called in the exitHandler. That tells me most of my other logic dealing with input/output streams should probably be inside of the startHandler. I also see I'll need to get the stdout of the process to print it. So I start with this simple code and see where I can get from there (note for security reasons I've changed the community code, and IP of the device).
#library('snmp_poll');

#import("dart:io");

void main() {
  var proc = new Process.start('/usr/bin/snmpget', ['-v 2c', '-c public', '127.0.0.1', '.1.3.6.1.2.1.1.1.0']);
  proc.startHandler = beginHandle() {
    var res = proc.stdout;
    print(res.read());
  };

  proc.errorHandler = errorHandle(errMsg) {
    print(errMsg);
  };

  proc.exitHandler = exitHandle(exitCode) {
    print(exitCode);
    proc.close();
  };
}
null
1

Initially I had some issues with assigning the handlers but that's because I tried to use the startHandler, errorHandler, and exitHandler as methods as opposed to properties with get/set. That was simple enough to straighten up so didn't bother showing that mistake here.
So after reading the API for InputStream, I see I should retrieve the data with the read method. But all I'm getting is null. Also, my return code is 1 which tells me it didn't execute properly. The process itself did execute which is why it's not calling the errorHandler. But still there's something missing. No output and why am I getting a 0 for the return code. Time to add some more code and a few debugging prints.
#import("dart:io");

void main() {
  var proc = new Process.start('/usr/bin/snmpget', ['-v 2c', '-c public', '127.0.0.1', '.1.3.6.1.2.1.1.1.0']);
  proc.startHandler = beginHandle() {
    var res = proc.stdout;
    var err = proc.stderr;
    
    print("Inside beginHandle");
    print(res.read());
    print(err.read());
  };

  proc.errorHandler = errorHandle(errMsg) {
    print("Inside beginHandle");
    print(errMsg);
  };

  proc.exitHandler = exitHandle(exitCode) {
    print("Inside beginHandle");
    print("Exit code is: $exitCode");
    proc.close();
  };
}
Inside beginHandle
null
null
Inside beginHandle
Exit code is: 1

Okay so now both stderr and stdout are both showing null. Gah what's going on. I've done something wrong obviously. Reading the API for InputStream tells me that if no data is available null is returned. But why is there no data available. Standard out or Standard error should be reporting something here.. All I can think is that we need to use the dataHandlers because these are streams rather than dumped to a string as I'm used to with other scripting languages (particularly when using the back-tick notation). Alrighty then, lets add the dataHandlers.
#import("dart:io");

void main() {
  var proc = new Process.start('/usr/bin/snmpget', ['-v 2c', '-c public', '127.0.0.1', '.1.3.6.1.2.1.1.1.0']);
  proc.startHandler = beginHandle() {
    print("Inside beginHandle");
    var res = proc.stdout;
    var err = proc.stderr;
    
    res.dataHandler = nowRead() {
      print("Inside stdout dataHandler");
      print(res.read());
    };
    
    err.dataHandler = errRead() {
      print("Inside stderr dataHandler");
      print(err.read());
    };
  };

  proc.errorHandler = errorHandle(errMsg) {
    print("Inside beginHandle");
    print(errMsg);
  };

  proc.exitHandler = exitHandle(exitCode) {
    print("Inside beginHandle");
    print("Exit code is: $exitCode");
    proc.close();
  };
}
Inside beginHandle
Inside stderr dataHandler
_InternalByteArray
Inside stderr dataHandler
_InternalByteArray
Inside stderr dataHandler
_InternalByteArray
Inside stderr dataHandler
_InternalByteArray
Inside stderr dataHandler
_InternalByteArray
Inside beginHandle
Exit code is: 1

Alright now we're getting somewhere... except apparently print doesn't like the output from the read method. Though looking at the API I do see that it does return a List. So lets iterate over that list to see what's in them. Since I see its only the stderr reporting the results I'll just play with that one now. And to save some screen space I'm just going to paste the snippet below:
...
err.dataHandler = errRead() {
  print("Inside stderr dataHandler");
  for(var x in err.read()) {
    print(x);
  }
};
...
Inside beginHandle
Inside stderr dataHandler
73
110
118
< ... snipped for brevity ... >
50
99
10
Inside beginHandle
Exit code is: 1

Ack! Okay that's so not what I wanted.. I'm getting integer character codes instead of the strings or characters returned. I need to convert these character codes to a new string. Fortunately a little search found just the constructor I needed in the String class. new String.fromCharCodes. So a quick little adjustment to my script and I can actually see what error I'm receiving from the snmpget command itself. And I no longer require that silly for loop.
#import("dart:io");

void main() {
  var proc = new Process.start('/usr/bin/snmpget', ['-v 2c', '-c public', '127.0.0.1', '.1.3.6.1.2.1.1.1.0']);
  proc.startHandler = beginHandle() {
    print("Inside beginHandle");
    var res = proc.stdout;
    var err = proc.stderr;
    
    res.dataHandler = nowRead() {
      print("Inside stdout dataHandler");
      print(new String.fromCharCodes(res.read()));
    };
    
    err.dataHandler = errRead() {
      print("Inside stderr dataHandler");
      print(new String.fromCharCodes(err.read()));
    };
  };

  proc.errorHandler = errorHandle(errMsg) {
    print("Inside beginHandle");
    print(errMsg);
  };

  proc.exitHandler = exitHandle(exitCode) {
    print("Inside beginHandle");
    print("Exit code is: $exitCode");
    proc.close();
  };
}
Inside beginHandle
Inside stderr dataHandler
Invalid version specified after -v flag:  2c

Inside stderr dataHandler
USAGE: snmpget [OPTIONS] AGENT OID [OID]...
<... snipped for brevity ...>

Alright so I'm getting the full snmpget error message but at the top I can see it's not happy about my flags. After playing around a little I discovered that this has something to do with how exactly the Process.start passes the argument list. The two options I came up with were to remove the spaces (which can sometimes cause issues with the receiving program) or after each flag place its argument as its own member of the list. So making that change I end up with the following code which works perfectly and calls the snmpget command and prints its return.
#import("dart:io");

void main() {
  var proc = new Process.start('/usr/bin/snmpget', ['-v', '2c', '-c', 'public', '127.0.0.1', '.1.3.6.1.2.1.1.1.0']);
  proc.startHandler = beginHandle() {
    print("Inside of start Handler");
    var res = proc.stdout;
    var err = proc.stderr;
    res.dataHandler = nowRead() {
      print("Inside stdout dataHandler");
      print(new String.fromCharCodes(res.read()));
    };
    err.dataHandler = errRead() {
      print("Inside stderr dataHandler");
      print(new String.fromCharCodes(err.read()));
    };
  };
  proc.exitHandler = exitHandle(exitCode) {
    print("Inside of exit Handler");
    print("Exit code is: ${exitCode}");
    proc.close();
  };
  
  proc.errorHandler = errorHandle(errorMsg) {
    print("Inside of Error Handler");
    print(errorMsg);
  };
}
Inside of start Handler
Inside stdout dataHandler
iso.3.6.1.2.1.1.1.0 = STRING: "Super Top Secret Equipment Name"

Inside of exit Handler
Exit code is: 0

Finally! Success! This is just the start of the script which I can now use to handle, parse and interpret the results of the snmp queries I make and pass them along as needed, perhaps providing them via json as an ajax service. In any case I certainly learned a lot more than expected about how Dart handles I/O and how I, in turn, need to handle the requests. While this small sample in itself seems rather verbose I can see how a few small wrappers could make dealing with a large number of processes fairly clean and robust.

Library hide and seek

So today I took some time to start playing with the IO library in Dart. Right off the bat I started running into problems. First off, I'm running Linux 64-bit and I'm using the latest integrated version of the IDE and SDK (Build 3934). Taking a look into the SDK I do see the IO libraries and such so I couldn't wait to get started. My first little script just spawns a new process running and displays the results. In this case in particular I'm spawning an snmpget command to run a quick and simple snmp query.

So I fire off my quick and dirty little script to see if I can get any output. I save my file and... err what what? The editor is giving me a nasty little error. Right there on line one of all places (my library declaration), a few other errors follow as well.
File not found: dart:io

A little further down is:
Multiple markers at this line
  - no matching constructor for Object
  - cannot find type Process

This line is where I make my new Process object using new Process.start(). Based on the first error, I gather that this line is generating an error because the IDE isn't recognizing any class called Process for me to create a new object from so it's trying to use the base Object class, which doesn't contain a named constructor called 'start'.

I mess around in the IDE to verify that the SDK libraries are available and that it is seeing the SDK as installed. Yeah, everything seems hunky-dory there. And... wait... my library panel is displaying the dart:io library right there in the left-hand pane. Something here doesn't seem right at all. I do a quick search in the dartbugs but don't find anything. So I came up with the following quick trial case and submitted a bug report.
#import("dart:io");

void main() {
  print("Hello World");
}

Save the above in the IDE and you should see the error generated at line 1. As I mentioned, I filed a bug report: Issue 1577. At first I thought that would be the end of my trials today, however I hit the run button anyways and low and behold the console is showing expected output. So while the IDE isn't recognizing the library in the editing pane, it is recognizing it in the libraries (though showing errors in it none-the-less) and the dart VM does run the code as expected. Excellent maybe I will get to do some testing after all!

Continued in INs and OUTs of IO.

Monday, February 6, 2012

New Editor and SDK

Just today I was thinking "There's been a large number of changes to the SDK lately and the integration builds of the SDK and Editor haven't been updated. Maybe I'm going to have to start trying the continuous builds..." when low and behold, a thread on the Dartlang misc group indicates that there is a new integration build available. Build number: 3934. So off I go and download that and the new SDK build.

Beautiful. First a couple of things to note: I extracted the editor into the dart/ directory it was in, into my home directory. I extracted the SDK into my home directory as well and left it in the dart-sdk directory. I launched the Editor and in the launch preferences pointed the VM to the /dart-sdk/bin/dart executable. Which worked fine, but the SDK and compiler preferences said they didn't see the SDK installed and thus indicated that frogc would not work correctly. Then I recealled that I had seen somewhere previously in the mailing group that one should extract the dart-sdk into the dart editor directory. So I moved the dart-sdk directory into dart/, relaunched the editor and low and behold under the launch preferences it says there are no launch settings available... that's interesting. I then go in and check the SDK and Compiler preferences. It shows that I have SDK build number 3934 installed. Yay. It's seeing it. But odd there's no launch preferences.....

Or so I thought. Wow. Look at all those options... I opened up a small test file I had from the past and click the little drop down arrow next to the Run button... maybe they added the checked mode flag I had mentioned before. Whoa.. what's all this? Manage launches and I have a nice little dialog to "create an manage launches". Hitting the new button I see I can create a Dart launch in Dartium (don't have that installed at the moment but that's alright, Dart Server launch (primarily what I use for dealing with the dart VM directly) and Dart Web Launch. I go ahead and create a new server launch. Give the launch a name 'Run Test' in this case. Hit browse from the script and choose the only script in this project anyways. But its interesting that it lets you choose which file is the one that contains the main method in such an easy manner. And it appears (though I've yet to test it) that it will only list files associated with your current project). Any arguments I want to pass to the script as well and finally to run in checked mode or not. Wow very nice. I go ahead and create a separate launch just for running in checked mode as well just because I can. I must say I really like that launch interface in this new editor.

And now the SDK also has the IO libraries available to me as well so I will soon have to start playing with these to work on some of the scripting aspects I'm interested in using Dart for, server side.

Thursday, February 2, 2012

Few Updates

So I haven't had the opportunity to write much lately. There's been a lot going on a work and home that have made my time far too sparse. Hopefully I can start getting some of that stuff settled soon. In the mean time there's been lots going on with Dart for me to get caught up with.

Firstly, we're starting to see some server side related API's coming out. Including file and directory libraries and the start of some networking and socket libraries. They are only available in the continuous builds at the moment, however I expect to see them in the integration builds soon (speculation not based on any actual evidence). These are something I was most looking forward to, for both server side scripting and even just generic scripting possibilities. I think in the near future I will try to convert some of my existing applications and scripts from their various options (Javascript, PHP, Ruby) to Dart to see what issues I run into.

With the addition of the new libraries, we see a mass of updates to the current API documentation. Thanks to Mozilla's work documenting the DOM (And releasing it as CC) much of this information is now available in the Dart APIs. Check them out and become enlightened!

Speaking of becoming enlightened, if you have played with Javascript in the past, or perhaps are a bit of a Javascript Guru, check out the Dart Synonym page. Lots of little snippets that show how you accomplish something in Javascript and how you would do the same in Dart. It even includes a small subsection on comparisons between JQuery and Dart.

Also, be sure to check out the new Dart news and Updates site.