Wiki Navigation

Dead Page!

This page has been made irrelevant by XMLLogFormat. It is preserved for posterity.

Note: an SQLite Logger was added in [13358]

Requirements

Whatever log format we change to must have these properties:

It must be indexable by Metadata (Spotlight) (Addresses #265)
It should be something we can easily (quickly, painlessly) turn into pixels on the screen.
It should not have less information than the current logs (logging display name only, for instance) (Gaim fails this)
Not all pieces are important -- display names, for instance
It should be easily parseable by a machine (Gaim fails this for text)
- Clearly tell which account is which (Gaim fails this for text and gets a D for HTML)
It should be easily parseable by a human
It should be used for message history (aka context). This mostly requires being able to read the last few messages without parsing the entire log.
Should have date stored in a format which can be freely localized (#542)

Bonus points for…

It should be easily searchable outside Spotlight (e.g. in the Log Viewer).
It should be something we can use on Panther without much pain.

evands: If it means a better system, I see no reason we can't do like Mail.app and use Spotlight for our searching needs. 10.3 users can continue with the current system. Caveat emptor: Will need to find a 10.3 way of doing message history.

This was in discussion of evands comments on irc:

15:18 <@The_Tick> Catfish_Man: this is the same reason taht 10.2 started really sucking for us
15:18 <@The_Tick> we used different things for each platform
15:18 <@The_Tick> and then we stopped caring about previous stuff
15:19 <@The_Tick> Catfish_Man: so if we do that
15:19 <@The_Tick> we might as well drop 10.3

Our focus is already on 10.4... given a choice between no progress and progress for 10.4 only, our choice is obvious. -evands

Notes

The Excelsior! framework might or might not be useful for XML-based formats. It's under the creative commons attribution license.

Proposals

Please use the following format:

Format name

Description of format or link to a specification or examples.

Proposed by: Bob

Alice: This sucks!
Bob: This rocks!
Charles: This sucks rocks!

XML containing XHTML

not too much different from the existing format. a log would look like this:

<!DOCTYPE ...>
<adiumlog>
	<entry date="2005-04-01 12:00:00 PM -0800" account="AIM.bob" recipientdisplay="Bob Roberts" sender="AIM.alice" senderdisplay="Alice Alisson">
		<html xmlns:...>
			<body>
				<p>I love <a href="http://www.adiumx.com/">Adium</a>'s new log format!</p>
			</body>
		</html>
	</entry>
</adiumlog>

(whitespace added for clarity)

proposed by: Mac-arena the Bored Zo

Catfish_Man: I think something like service="AIM", sender="alice", account="bob" instead of the above would be both more readable and more flexible (replacing searches like account contains '.alice"' with sender equals 'alice' just seems friendlier)
Mac-arena the Bored Zo: this does kind of violate the proper separation between structure (HTML) and style (CSS). if we do something like this, we should have a way to provide CSS with it. one way might be to supply the CSS from within Adium; another way would be <span style="..."> tags. I don't think <style> would work for this - it would make the logs too big.

jmelloy: This seems like the best solution, but is way too verbose. Logs will be enormous. Something like this would be better, I think:

<!DOCTYPE ...>
<adiumlog date="2005-04-01">
	<message time="12:00:00 PM -0800" account="AIM.bob" sender="AIM.alice" recipient="AIM.bob">
		I love <a href="http://www.adiumx.com/">Adium</a>'s new <b>log</b> format!
	</message>
         <status time="12:05:00 PM -800" account="AIM.bob" type="away" message="Now is the winter of our discontent">
                 Bob has gone away with the message "Now is the winter of our discontent"
         </status>
         <status time="12:05:47 PM -800" account="MSN.jimmy@hotmail.com" type="display" message="bleh">
                   jimmy@hotmail.com has changed his display name to "bleh"
         </status>
</adiumlog>

Catfish_Man: Here's my latest variation on the XML theme. This is, in certain ways, less flexible: it doesn't store style information nearly as much. The lack of style info is just because I have yet to find a space-efficient way of storing it. The basic idea of the changes is to trade having more tags for being more space efficient.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//AdiumX//DTD LOG 1.0//EN" "http://www.adiumx.com/DTDs/Log-1.0.dtd">
<log version=1.0>
    <chat date="2005-04-01" service="AIM" recipient="alice" account="bob">
        <inStatus time="12:05:47 PM -800" type="display" message="bleh">
            alice has changed her display name to "bleh"
        </inStatus>
        <outMessage time="1:00:00 PM -0800">
            I love <a href="http://www.adiumx.com/">Adium</a>'s new <b>log</b> format!
        </outMessage>
        <inMessage time="2:00:00 PM -0800">
            Yeah!
        </inMessage>
    </chat>
    
    <groupChat date="2005-04-01" service="AIM" account="bob">
        same stuff here, just we have to attach a "sender" property to each inMessage, since there can be more than one.
    </groupChat>
</log>

jmelloy: Here's a recent attempt, which can be seen in action at http://www.visualdistortion.org/misc/xmlViewer.xml:

<chat account="fetchgreebledonx" service="AIM" recipient="resimada" date="2005-05-21">
    <outMessage>
        <time>11:06:14</time>
        <sender alias="Jeffrey Melloy">fetchgreebledonx</sender>
        Here, check this out: <a href="http://www.visualdistortion.org/misc/xmlViewer.xml" title="http://www.visualdistortion.org/misc/xmlViewer.xml">http://www.visualdistortion.org/misc/xmlViewer.xml</a>
    </outMessage>
    <inMessage>
        <time>11:07:15</time>
        <sender alias="Adam Iser">resimada</sender>
        cool
    </inMessage>
    <inMessage>
        <time>11:07:23</time>
        <sender alias="Adam Iser">resimada</sender>
        postfix a : after the sender name and some ()'s around the time! :)
    </inMessage>

This format uses CSS to style the text however we want it. Downsides: It is very tough to see the alias with pure CSS.
<sender>resimada</sender>
<senderAlias>Adam Iser</senderAlias>

Mac-arena the Bored Zo: I think we should use XSLT (or pure Cocoa) to transform the XML instead of putting message attributes (such as time) into sub-elements like that. when did libxslt start coming with the OS? how practical would it be to bundle it? (it's MIT-licensed, but we do have the size of the bundle to worry about.)
Mac-arena the Bored Zo: whatever XML format we use, we can use the LMXParser.

Property List

an array of dictionaries, with keys such as:

Sender (an account specification, e.g. "AIM.alice")
Sender display name
Recipient (an account specification, e.g. "AIM.bob")
Recipient display name
Date (and time)
Message data format (UTI: public.plain-text, public.html, public.rtf, ...)
Message

proposed by: Mac-arena the Bored Zo

Mac-arena the Bored Zo: problem with this is, it wouldn't be all that pretty and it could get big very easily. binary plist format might help with the latter, though.
Catfish_Man: see comment above on handling accounts/services/screennames
jmelloy: ugh

Trillian

(that is, Trillian)

initially mentioned (not proposed as such) by: Ksilebo

Mac-arena the Bored Zo: [barfs] HTML in an XML attribute... [barfs again]

Promote the SQL Logger to being the default logger

if we do this, though, we'd need to either bundle PostgreSQL, or port to SQLite and bundle that, being that no SQL database engine comes with Panther.

Bundling PostgreSQL seems like more trouble than it's worth
There would be speed hits in some places users would not expect ... since the main use is through a web interface, it's hidden in the reload time of the page.
- when you click a name in the drawer and get their list of logs.
- when you click a name in the window and load the log itself

Advantages:

Show multiple conversations at the same time
finer grained searching

Proteus seems slow with SQLite, but their schema is stupid.

Note (-samsamoa): I've succeeded in compiling sqlite3 into the Adium binary. And now I've implemented the logger too! w00t. Now I just need the viewer. wheeeeee

proposed by: Mac-arena the Bored Zo

Other iODBC-compatible database

this would work on Panther, but we'd still need to bundle a database engine to back iODBC with.

jmelloy: I don't see why this would be an advantage over PostgreSQL, unless you wanted to use the iTunes V100 database or whatever.

proposed by: Mac-arena the Bored Zo

ASCII database format

something that separates each log entry with either ASCII record separator (U+001e) or some combination of CR (U+000d) and LF (U+000a), and each component of the entry with ASCII field separator (U+001c) or horizontal tab (U+0009).

proposed by: Mac-arena the Bored Zo:

Mac-arena the Bored Zo: FS and RS isn't human-readable.
Mac-arena the Bored Zo: we would probably need to prefix each field with e.g. 'Display name: ' for future-proofing, unless we want to refer to the fields by index only (which is bad).

proposed by: Mac-arena the Bored Zo

Apple System Logger

reference: <asl.h>

this API looks like it does everything we need (except for being human-readable).

proposed by: Mac-arena the Bored Zo

advantages:

straightforward.
already exists: no need to create a new log system.
does everything we need.

disadvantages:

requires Tiger.
~~not human-readable.~~
All log entries go into the central ASL database as well as our log file.
YA log format.

Download in other formats:

Plain Text