Cafe Bucky

As I have recently written, I’m working on getting a self contained environment together for running OpenSim and LAMP services on a Mac.

In other words, running a virtual world on your Mac off of a USB key.

So far, so good. But … I’m stuck!

I need some help in getting OpenSim and Mono configured. I’m running into a showstopper problem when a user logs in from a viewer. I’ve decided to put up a snapshot of what I have so that some experienced eyes can take a look at the problem.

The Pre-Alpha snapshot is about 240 meg, and you can get it right now.

The snapshot is a Mac disk image (a compressed sparse bundle). Download, uncompress, and mount. It should mount as /Volumes/MacSimStick. All of the main components are under /Volumes/MacSimStick/usr/local.

See the top level README for ports, passwords, layout, etc.

So what does work?

Apache, PHP, MySQL, and the Diva Wifi interface are all doing fine. There are some configuration tweaks to do, and lots of files and directories that can be removed. The goal right now is to get the end to end functionality sorted, and then we can go back and prune & tune. There is a simple minded shell script, Start.sh, at the top level. I expect that anyone toying around with this at this point is going to be in a terminal window, and is comfy with launching apache, mysql, opensim, etc from a command line.

What is the problem?

The problem seems to be something about how Mono deals with the network. I provided a full logging of the problem in /Volumes/MacSimStick/Logs/OpenSim.log. That file shows the full run of Configure.exe, followed by a session of OpenSim.exe.

Some suggested ways to attack this …

Use your own version of Mono against the diva distro in MacSimStick. If that works, then we know we have a problem specific to my install of Mono 2.8.

Did I miss anything in the Mono or OpenSim config files? (probably!)

Could the problem have something to do with virtual network interfaces (such as the one created by Parallels?)

Is the version of Mono just fine, but we find that it works with an older OpenSim distro?

For reference, my Mac is a 2006 Intel iMac running 10.6.4. I have tried the phoenix and imprudence viewers, and I get the same problem each time. At least that part is consistent.

What are the next goals?

Well, once we have something that functions much like the Windows USB counterpart, I will turn my attention to:

• Simple Startup and Shutdown
• Getting rid of excess files and directories
• Tuning of all config files
• Making use of the web server. Provide info on where to get viewers, a local wiki for the project, and be able to serve as a web gateway for objects in world that need it.
• Getting a good logo

Some more goals:

• Be completely self contained (not touch the local disk at all.. I do currently use a MySQL socket in the standard /tmp location)
• Allow for different mount points besides /Volumes/MacSimStick (am not sure if config files can take care of that for everything…)
• Make it easy to change ports and passwords

I think this is all doable. Although one part of me would love to hold off on snapshots until I have everything working end to end, it’s more important to know when to ask for help! That way, those of us on the Mac side can get going on this that much quicker.

Thanks for any help!

Recently some enthusiasts got together the idea of running OpenSim off a usb key. Even doing a video to show the process.  Pathfinder did a great post on it, as did Ener Hax.  Tateru Nino made a ready to go version of it available.

Pretty Cool, if you are a Windows person.  You plug in a usb key, press a couple of buttons, and have a self contained virtual world on your machine that you can connect to with your fave viewer (SL, Phoenix, Imprudence, etc.)

I’d been waiting to see if someone would do the same thing for the Mac folks.  Time Passed …

… Well … ok, fine, I’ll do it!

So I am undertaking the MacSimStick Project.

I must be out of my mind.  There are a lot of moving parts :)

So What Is The Goal?

The goal is to insert a usb key into a mac, click on an icon, have a self contained virtual worlds environment come to life, use it, and to be able to walk away with any changes you make.

That nicely glosses over about a dozen things happening in the background.

Some assumptions and goals:

• be self contained, keeping off of the local disk as much as possible (log files, etc)
• dont make the user install anything
• wont assume what they have installed (such as Mono).
• dont interfere with web server or database already running on the machine (use different ports)
• provide the power to change things (MySQL and OpenSim consoles), but work without requiring their use

What Are The Puzzle Pieces To This?

• Apache 2.2.17
• MySQL 5.1.52
• PHP 5.5.3
• Mono 2.8
• OpenSim – Diva Distribution

A reason why I include Apache and PHP is because it opens up a lot of interactively options for scripts in a Sim to call into a local web server.

Where Am I With It?

I am working with a disk image on my iMac:  /Volumes/MacSimStick

The first iteration of this will depend on that name.  I compiled Apache, PHP, MySQL, and Mono to depend on paths within /Volumes/MacSimStick/usr/local

Apache, PHP, and MySQL work fine.

The Diva Distro of OpenSim is connecting to MySQL and writing to the database, but bombing out further in the process.  I’m thinking that a good approach will be turn off a lot of things (why is it trying to call JScript?), get things working end to end, and then slowly turn some things back on.

If I am still stuck in a day or so, I will upload what I have to get some more eyes on the problems.

What’s Next?  What Do I Need?

As far as configuration goes:

• corral all of the config files together by having a Config dir full of links to the actual files (PHP, Apache, MySQL, Mono, OpenSim)
• create a startup script which launches a terminal session (using the “screen” command).  This will provide separate shells for MySQL and Opensim.
• create a shutdown script

I need beta testers.  I dont want to make this widely available until I am confident that it works for a handful of folks (for one thing, it’s going to be a sizeable download).  Contact me at my gmail.com address: javajoint, and I will point you at a download.  I am pondering if I want to put this up at Amazon S3 or Dropbox …  It’s going to be a few hundred meg.

I need some extra eyes checking out the configuration.

Donations!  This is going to cost me in hosting, and I am between jobs.  If you can help sponsor the work, cost of hosting, etc, I’d sure appreciate it.  I’ll stick a PayPal button at the bottom of this post.

More on this soon.  Thanks for the enthusiasm so far!

Not long ago, I wondered aloud about what to learn.  Even if I stopped sleeping, I’d still not have the time to absorb all of the things I’m interested in.  I’m glad that I will never be bored, but I am always dealing with the priorities of “what to focus on?”  It’s a fun side of being a generalist: so much is so damn interesting!

So I thought I would write about the flip side, and mention what I am learning.  That way, potential employers and friends will know that I am not sitting around cooking bacon for my cat all day, and that I do have a project or two.

I am focusing on Cloud Computing, C# and .Net/Mono, NoSQL, OpenSim, and Unity.

They all tie together too, kinda.

As you know, a big interest of mine is Virtual Worlds.  I’m a champion of OpenSim and the Hypergrid, because I see a lot of parallels to where we were with Apache and the Web in 1995.  I am glad we have the Windows Sim on a Stick download which utilizes the Diva Distribution.  I am  currently looking into how to use MAMP to do the same thing for the Mac community.

Now that I have mentioned the server side of things, let me do a quick take on the client.  As I’ve written, I really want to see a Unity-based front end to OpenSim.  I have started porting my Slider object from SL to Unity.   My work so far has been in JavaScript, but in order to network it with, say,  Smartfox Server, I’ll be writing a bit in C#.  I see some good reasons to learn C# and Mono (starting within the contexts of Unity and OpenSim [1]).   I’ve been reading Programming C# 4.0.

[1] Seeing as how OpenSim is written in C#

So far so good.  You may be wondering how I tie Cloud Computing into this.  On the Cloud Computing side, I’ve been a user of Amazon S3 for a while.  EC2 was sorta off my radar, but a recent post by Ener Hax caught my attention, where she points at the idea of using the cloud to host OpenSim instances.  This put me on the chain of thought of “how to quickly scale for events in Virtual Worlds?”.  That has me delving into Amazon Web Services, and I am reading Jeff Barr’s Book “Host Your Web Site in the Cloud“.

The last bit of my current techy study stuff is NoSQL.  I’ve been through the try-redis tutorial and love it.  I am decent with MySQL, but I’ve always been a big fan of hashtables/associative arrays/keys & values etc… it fits my mental model of what I like to do in a program (index into an array, based on a key) — having that approach for a database makes sense for a couple of projects I have in mind (my photo database, and ThereThen addresses) .

So that’s my current focus.  I see jobs come across that want other things (like Java…), but I like what I’ve picked for myself!

I was just reading an article on MakeUseOf.com: Create A Google Custom Search Engine For Your Research

I got inspired to tinker with a custom search engine for Virtual Worlds stuff.  Let me know if you want to collaborate on it.

Try a sample search. Go on. You know you wanna ;)

I’m in a bit of a Quandry.

I’m in all out job search mode, and I get a fair number of inquiries from recruiters each day.  They seem to like my resume.

So far, so good.

The thing is, I do know a wide range of tech, and much of what I see fits me well, but there is often some skill that I dont have, such as some language or some bit of technology.   Either I dont know RoR well, or .Net, or am rusty on Java, etc.

There is so much that I could learn, but there is only so much time.

So that is the fun part.. I am a great generalist, and am interested in a lot of facets of Tech.  So what to focus on?  It’s no good changing gears every day.

My idea at the moment is:  I’ll jot down the things I am very interested in learning. and hopefully a couple of folks will write me and say “well hell, Daniel, just dont sleep — you mean to say you dont know all of this already!?”

Languages and Frameworks

• PHP – I know 5.x, and a bit of CakePHP.  So, Zend Framework?  Symfony?  CodeIgniter? Others?
• Python / Django? – I’ve gone into python and have made minor changes.  I like Python from what I know of it.
• Ruby / RoR – I’ve done some Ruby on Rails tutorials, and dug it.  I have not had a project to really get my feet wet with it.
• Perl – actually, I am pretty good wth Perl.  I  just have not touched it much since 2006 at Friendster.  I actually wrote a perl shell back in 1992.  Catalyst Framework?  Mason?
• C# / .Net / Mono – this may come as a surprise to anyone who knows my leanings to all things Mac and Linux.  My interest comes from wanting to do more with two key projects:
  • OpenSim – yep, OpenSim is written in C#, and runs on .Net or Mono
  • Unity – the scripting languages for Unity (JavaScript, C#, and Boo) run on top of Mono, and access underlying .Net libraries.  I also love the idea of Embedding Mono for other projects.

In the category of “well, if I need to go back to…”:

• Java -  I have not done new java code since 1996!  I would not want to work on client side java at this point (who is doing applets at this point?)  — I would be into the whole Eclipse + server side angle (I’ve worked a bit with Weblogic and Presto)   Java has so many technologies associated with it at this point, that it is hard to know where to jump in.
• Flash / AS3 -  I have not touched flash much since 2005.  I think HTML5 + JS + CS3 is the way forward.  I do like ActionScript, which is a close cousin of JavaScript.  When I dealt with AS2, I had one entry point on the first frame and used a lot of classes.  I didnt do the snippets attached to the timeline deal.  My leaning is that I could back into flash again for a specific project, but there are many other things I’d like to learn first
• C / C++ – It’s been forever ;)

Content Management Systems

Ok, which to learn?  Drupal?  Joomla?  Wordpress?

NoSQL / SMAQ

I’ve been reading a lot about NoSQL, and just came across this article about SMAQ (Storage, MapReduce and Query)

What to learn?  Hadoop?  CouchDB?  Cassandra?  Voldemort?   etc…

Besides Programming

It’s obvious I am interested in a wide swath of the programming side.

Now, if someone were to ask me what I would do besides programming, I would say photography, writing, or possibly speaking / demoing.

The other big interest of mine is multimedia.  I know a bit, but wish I could just dive into
it for a year and learn a lot more.  Some apps I would like to know would be:

Logic Studio
Final Cut Studio
PhotoShop CS5
Maya / 3DS Max /  Blender

A lot of the output of this would be directed towards work in Virtual Worlds (assets in
a Unity project, as one example)

(in other words, I know a bit of the creative side, and wish I had the time/money/chance to
really learn a lot more of it)

Anyways, to sum up, I’m interested in a lot, but every situation wants different skill sets.  Hard to know where to focus!

Overview

I’m a longtime SL user (March 2006), and I think the 1.x and 2.x codebases are becoming an evolutionary dead-end.  I think the community will face a day of reckoning, and developers will show a way forward.  In some ways, this mirrors the situation of the LL server versus OpenSim.  The Unity development tool would seem to be a very attractive avenue forward.

The server side has been reinvented by OpenSim.  The client side needs a huge reset.

1.x and 2.x Codebases

The current situation with SecondLife / OpenSim viewers is that we have two codebases: 1.x and 2.x. The overwhelming majority of users are on some variant of the 1.x codebase (SL Viewer,
Imprudence, Phoenix).  The 2.x users are, for the most part, on the SL Viewer 2, or on Kirstens Viewer.

Another factor in the current situation is that Linden Lab has put all development effort into 2.x.  They know that the UI issues for the 1.x users are pretty significant.   Since most of the community doesn’t want anything to do with the 2.x interface, Linden Lab will have a hard time attracting much unpaid developer effort to their side.

The other trend affecting LL is that they have some features coming up that will require the 2.x viewer (meshes come to mind).   It’s an unfortunate mix of variables.

For some time to come, the 1.x viewer community will thrive.  They have made significant progress in giving the community what it wants (double click teleports, area search, inventory mirroring, better LSL editing options, more convenient Windlight UI access, just to name a few).  If LL were to pull support for 1.x viewer access [1], it would backfire in a big way — it would be an event that would lead to mass migration.  Whether they like it or not, they are somewhat forced to keep the 1.x viewer community connected.  They have clearly painted themselves into a corner.

[1] nah, they wont — I have a backchannel on this, as well as what was stated on The Developers List and also this one

In short, the 1.x UI is clearly the winner, but on the server side, the way forward will require 2.x.  The LL server / client situation is messy.

… And… neither viewer stream could hope to compete with an AAA level game engine within, say, the next year or two.  My concrete example is that the Unity engine is far ahead in many areas.

Enter the Unity Game Engine

What of true advancements?  What would have to happen to the viewer to bring it up to the standard of a AAA game title?

Looming on the horizon, there are significant walls for the 1.x and 2.x codebase.  To explore but a few areas, what can be done about Graphics, Animation, Sound, and Terrain?

Let’s focus on the positive for a bit :)

Graphics – A quick look at Unity demos (for instance, Tropical Paradise,  Shadows, and Butterfly) gives a strong hint as to its power.  Observe more lights in one scene, dynamic shadows on older hardware.  Custom shaders that can be written by 3rd parties.   Unity 3.0 will be released soon, and goes even further:

• Deferred Rendering — Dark Unity
• Beast Lightmapping — Lightmapping in Unity 3
• Image Effects

Physics – Physics in Unity are very advanced:

• http://unity3d.com/unity/features/physics
• cloth physics in 3.0

Animation – Unity has advanced object and character animation:

• game objects
• locomotion extension
• animation blending GUI

Sound – so much control as to mixing and where the Audio Listener is positioned

Terrain – very powerful from the built in editor, as well as from scripting, and a very well done extension (Terrain Toolkit)

Editing (3.0) – Snapping & Marquee Selection — also note that there is a lot of scripting support (in JavaScript, C#, and Boo) which could be used to develop in-world editing.

Deployment – Here is the Overview for 2.6.  Unity is very well suited for Windows (Direct3D 9), Mac (OpenGL), the Web, Nintendo Wii, and the iPhone.  They are working on deployment for the Android, Xbox 360 and the PS3.

Where To Go From Here

It should be pretty clear that the 1.x and 2.x have little hope of catching up to something like Unity 2.6.  In the year or two it would take to get close, Unity and other game development engines will be that much farther along.  To me, it seems clear that there is a choice:

1. Keep advancing the 1.x and 2.x codebases, picking up some of the features of Unity, but still being limited to Mac, Windows, and Linux
2. Freeze current codebase (Linux will have to stay where it is…), and take the things about SecondLife and add them to an app built with Unity.  This will pick up additional platforms, and will tap into a much larger development community.

I note that there are at least two OpenSim + Unity projects underway:

• Resonance Live
• Heritage Key, by Rezzable

Enjoy the links, ponder it, and think about where you want to put your development efforts for the next 2 years ;)

When I was starting to learn PHP and MySQL in 2001, I wanted to tackle a project that meant something to me; a non-trivial web application…

So, being a photographer and programmer with some time on my hands, I decided to write a photo database.

The first iteration of FlexiPhoto had a hard-wired interface.  I rewrote it to support multiple UIs and languages.  Sometime in 2004-2005, I rewrote it again as a PHP5 app.

In 2006, I got distracted by Second Life :)  It was like going down a rabbit hole.  I got good at scripting in LSL [1].  I shelved my web projects.  What a mistake!  Although I have learned a lot of web tech in the last 5 years, I had my attention split by learning a lot about Second Life.  It’s nice to be good at it, but it doesn’t pay!  And, damn, I could use some money about now …

So I am back on the case with FlexiPhoto.  I have a virtual Ubuntu machine running at Slicehost, which I’ll use for the public-facing Demo, Wiki, and Download site.  I’m in the midst of dusting off FP (switching database layer from DB to MDB2, for one thing), and expect to have it available within a week.  It’s going to look clunky and very pre-AJAX, but then, I am a big fan of iteration.

A fun side note about Linux and running virtual machines at home: I still have my old 1U rack server, which hosted FlexiPhoto and other sites back in the day.  The damn thing is loud! It’s running an old version of Debian.  My task today is to mirror my Slicehost OS at home.  I’ve installed Ubuntu Karmic Koala (9.10) on my iMac, hosted via VirtualBox. That will give me synced development & public environments.  I also am able to dual boot my Samsung NC10 Netbook to Windows XP and Ubuntu.  I’ve got it covered, baby!

Geez, geeking out here big time.

Anyways, FlexiPhoto is a good portfolio piece of mine, and I am stoked to be working on it again.  It’s like an old friend :)

[1] …although, to this day, I really think Cory O should have finished LSL.  It even makes Tcl look good, and that’s saying a lot…

For many years, I didn’t believe in doing New Years Resolutions. I figured that it was a good idea to just resolve to change things as one went along. That’s still a good idea, but I do see the symbolism of the clean slate of a New Year.  Doubly so for a new decade.

And I also say Good Riddance to the decade of 2000-2009. There were lots of ups to it (I love you Kimmy, and you have filled our time together with so much love, energy, and grace — I want you in all of our decades to come :). There have also been way too many downs, and I’ve learned and just want to move on.

I’m blogging some of my resolutions, so that I put a little pressure on myself to get them done … I’m only listing things that I think I can really control. Stuff that involves other people or that gets a bit personal doesn’t get penned here.

Ze Resolutions

• Wake Up when the alarm goes off
  (hitting the snooze button on my iPod is way too convenient — I’m going to move it so that I have to get out of bed)
• Lose 20 pounds by June 1st
  (I refuse to go any bigger than 34×30 Levis 550’s — I am drawing my line in the sand!  task #1: buy a scale)
• Throw stuff out so that I dont need a storage area any longer

Tech & Creative Stuff

• Average one creative output per day (such as a picture, video, blog post, tweet)
• Make a list of all of the web sites I am registered with
  (and get rid of ones I am not likely to ever use again)
• Get daniel.org redone as a proper personal domain by February 1st (themed, pages for software and writings)
• actively use my lynda.com and safarionline.com accounts to keep learning (complete at least one video course per month, and read at least one tech book per month)
• Get slide scanner working and scan at least 50 slides per month
• learn final cut express and photoshop
• learn Ruby on Rails
• resurrect FlexiPhoto and implement ThereThen addresses so that photos can be aliased by time and location  — do this by March 1st.
• No more SL scripting unless there is a financial reason to do so
  (I feel like SL has been a huge detour — I am glad that I am very good LSL scripter and have done some innovative work, but I could have been using that time to do things that would have done more to pay the bills)

There are a ton of other things I could write as Resolutions, such as Get A Job or Travel To Europe, but anything that involves other people isn’t something I can completely control.   (I do think I will be working and traveling, but it’s not like 2000, where I had the resources to just grab tickets and friends and take off)

[update -- January 14, 2010]

So far, I am paying attention to some resolutions pretty well …

Kimmy – We’re not together any more.  I do wish her the best.

Waking up – Am doing much better at this!

Weight – have ordered a scale so that I know where I am at.  Lots of little tweaks, such as making 2 scrambled eggs instead of 3.

Creative output – yep!  Doing it.

Lynda.com and Safarionline – yep! Doing it.

Second Life – yep!  I have really pulled back and I dont script there any more.

The thought that sums up my mood lately is:

“I cant get the last few years back, but I can sure make the next few count.”

[update -- January 29, 2010]

Got a scale – doing daily weigh-ins, not gaining :)

Got a VPS (Virtual Private Server) via Slicehost.com and am configuring it

[update - June 1, 2010]

Averaging a creative output a day?  Close to it, if I count mix sessions I put up on Dropbox.

Slicehost working out well.  It hosts javajoint.com and my therethen* domains.