aboutsummaryrefslogtreecommitdiff
path: root/content/posts/2003-12-06-retreat.html
diff options
context:
space:
mode:
Diffstat (limited to 'content/posts/2003-12-06-retreat.html')
-rw-r--r--content/posts/2003-12-06-retreat.html63
1 files changed, 63 insertions, 0 deletions
diff --git a/content/posts/2003-12-06-retreat.html b/content/posts/2003-12-06-retreat.html
new file mode 100644
index 0000000..b5a44bb
--- /dev/null
+++ b/content/posts/2003-12-06-retreat.html
@@ -0,0 +1,63 @@
+---
+date: "2003-12-06T09:13:55Z"
+title: Retreat!
+---
+
+<p>
+I give up. I'm going to disable the parallel feed grabbing in <a
+href='http://www.raggle.org/'>Raggle</a> so we can put out a new
+version. <a href='http://www.pekdon.net/'>Claes (pekdon)</a> suggested
+I try and rewrite it, but the implementation is already pretty simple.
+Here's a high-level view of how the old non-parallel and new parallel
+feed grabbing stuff works:
+</p>
+
+<p>
+<b>Old Code</b>
+<pre>
+$config['feeds'].each { |feed|
+ # download feed
+}
+</pre>
+
+
+<p>
+<b>New Code</b>
+<pre>
+threads = { }
+$config['feeds'].each { |the_feed|
+ threads[the_feed['url']] = Thread::new(the_feed) { |feed|
+ # download feed
+ }
+
+ thread = threads[the_feed['url']]
+ if thread &amp;&amp; thread.status == 'run' &amp;&amp;
+ !$config['grab_in_parallel']
+ # thread.join
+ end
+ until Thread::list.size &lt; ($config['max_threads'] || 10)
+ $log.puts 'DEBUG: waiting for threads'
+ sleep 5
+ end
+}
+</pre>
+</p>
+
+<p>
+Of course, looking at this code as I'm pasting it, it just occured to me
+that if you have two feeds with the same <acronym title='Uniform
+Resource Locator'>URL</acronym>, you could have two threads trying to
+muck with the feed at the same time. Wonder if that's what's causing <a
+href='http://www.ruby-lang.org/'>Ruby</a> to freak out. By the way,
+this is why I <em>really</em> dislike threads. Not because I'm an
+ignoramus, but because they encourage subtle bugs like this. Anyway,
+let's see if that fixes our random crash woes.
+</p>
+
+<p>
+Oh, and before anyone asks, yes, I realize that's not the best
+way to implement the thread capping stuff. And yes, I realize thread
+pooling would be more efficient. Right now I'm just trying to get it to
+work reliably, then I'll focus on optimization.
+</p>
+