Go Language Resources Go, golang, go... NOTE: This page ceased updating in October, 2012

--- Log opened Tue Jun 07 00:00:33 2011
--- Day changed Tue Jun 07 2011
00:00 -!- Venom_X [~pjacobs@] has quit [Quit: Leaving]
00:02 -!- Tv [~Tv@ip-66-33-206-8.dreamhost.com] has quit [Ping timeout: 244
00:03 -!- dfc [~dfc@eth59-167-133-99.static.internode.on.net] has joined #go-nuts
00:03 < uriel> unit3: I'm not sure what you are doing, but you can also use
a single channel with many senders
00:04 < uriel> no need to use a single channel per goroutine
00:05 < unit3> uriel: yeah, that's what I was doing, but the problem back in
the loop in main() reading from the channel was that I didn't know when all the
subroutines were done, so the program would always exit with "throw: all
goroutines are asleep - deadlock!"
00:06 < uriel> unit3: I don't think "all goroutines are asleep" has to do
with not knowing if all goroutines are done
00:07 < uriel> just keep a counter of how many goroutines you start, and
count that you got as many results
00:07 < uriel> or use another channel to get notification from done
00:07 < unit3> problem is that each goroutine returns a nondeterministic
number of results.
00:07 < uriel> does each goroutine know when it is done?
00:07 < unit3> I'm hoping the WaitGroup thing will be a nice way to just
keep track of started vs done.
00:07 < Namegduf> So you can't wait until a certain number of results.
00:07 < unit3> Yeah, it does.
00:08 < uriel> then use another 'done' channel where goroutines send
something before they die
00:08 < Namegduf> First, define the desired behaviour.
00:08 < Namegduf> Then implement it.
00:08 < Namegduf> A done channel seems sensible.
00:08 < Namegduf> You can use select{} to listen on it and a "stuff to do
things with" channel.
00:08 < uriel> I'm not saying that WaitGroup is not the right tool, but it
is good to know how to do this things directly with channels
00:08 < unit3> Haven't looked at select{} yet, will now.
00:09 < uriel> i don't think you even need select
00:09 < Namegduf> In general, define in English under what conditions the
program should qit.
00:09 < uriel> (although you certainly should know how to use select if you
are writting go ;))
00:09 < uriel> you can start another goroutine to gather the results
00:09 < uriel> and let main do the listening on the 'done' channel
00:09 < Namegduf> Select is for if you want to get some results back and do
stuff with them, separately to the done channel, and don't want to make another
goroutine do that.
00:10 < uriel> Namegduf: if he is starting a bunch of goroutines already,
starting one more to gather results seems reasonable, but select is probably the
right way (or you would have to wait for the extra goroutine too anyway)
00:10 -!- araujo [~araujo@gentoo/developer/araujo] has joined #go-nuts
00:10 < Namegduf> Yeah.
00:11 < uriel> I'm just trying to mention various ways one can approach this
kinds of things
00:11 < Namegduf> The most generic solution is probably to have main() wait
on a channel sending a bool, and if given true increment the count, and if given
false decrement it, and return when the count hits 0; anything which wants to hold
the program running just sends a true, until it no longer wishes to, at which
point it sends false.
00:12 < Namegduf> But you can make a simpler thing for your program easily
00:12 -!- dreadlorde [~dreadlord@c-24-11-39-160.hsd1.mi.comcast.net] has joined
00:13 < unit3> Hmm...  let me see if I understand the select{} right.  given
multiple valid communications options, it'll just pick one at random to do...
00:13 < unit3> so if I have it wrapped in a loop with a counter, like you
said, and one of the options adjusts the counter based on t/f from the control
00:13 < Namegduf> It'll wait until at least one of the options is valid.
00:13 < Namegduf> Then do it.
00:13 < unit3> and another outputs from the output channel...
00:14 < Namegduf> If multiple are valid, then it's random.
00:14 < unit3> right.
00:14 < Namegduf> But that's not *usually* important.
00:14 < unit3> No, I think algorithmically it'll always complete putting the
output before it gets the final false from the control channel.
00:14 < unit3> just based on how I have it structured right now.
00:14 < unit3> That is easier than using a WaitGroup for my purposes, I
00:15 < unit3> I'll give it a shot, see how it goes.  Well, maybe tomorrow,
since it's nearly quitting time.
00:15 < unit3> Thanks so much for all the pointers and advice, guys.  Really
loving the Go community.  :)
00:15 < Namegduf> No problem.
00:16 -!- unit3 [~Unit3@] has quit [Quit: Leaving]
00:19 -!- dj2 [~dj2@CPE001f5b35feb4-CM0014048e0344.cpe.net.cable.rogers.com] has
joined #go-nuts
00:32 -!- wchicken [~chicken@c-98-248-194-46.hsd1.ca.comcast.net] has quit [Ping
timeout: 276 seconds]
00:33 -!- kr [~Keith@] has joined #go-nuts
00:45 -!- franciscosouza [~francisco@] has quit [Read error:
Connection reset by peer]
00:45 -!- Tv [~Tv@cpe-76-168-227-45.socal.res.rr.com] has joined #go-nuts
00:49 -!- marko_ [~marko@host194-207-dynamic.37-79-r.retail.telecomitalia.it] has
quit [Read error: Operation timed out]
00:49 -!- dreadlorde [~dreadlord@c-24-11-39-160.hsd1.mi.comcast.net] has quit
[Ping timeout: 252 seconds]
00:50 -!- cafesofie [~cafesofie@ool-18b97779.dyn.optonline.net] has quit [Remote
host closed the connection]
00:51 -!- cafesofie [~cafesofie@ool-18b97779.dyn.optonline.net] has joined
00:51 -!- dreadlorde [~dreadlord@c-24-11-39-160.hsd1.mi.comcast.net] has joined
00:54 -!- wchicken [~chicken@c-98-248-194-46.hsd1.ca.comcast.net] has joined
01:01 -!- kr [~Keith@] has quit [Ping timeout: 246 seconds]
01:02 -!- askhader [~askhader@taurine.csclub.uwaterloo.ca] has joined #go-nuts
01:08 -!- bmizerany [~bmizerany@] has quit [Remote host closed the
01:09 -!- foocraft_ [~ewanas@] has joined #go-nuts
01:12 -!- foocraft [~ewanas@] has quit [Ping timeout: 240 seconds]
01:15 -!- dlowe [~dlowe@c-66-30-116-162.hsd1.ma.comcast.net] has quit [Quit:
01:23 -!- crunge [~Crunge@about/security/staff/crunge] has joined #go-nuts
01:31 -!- franciscosouza [~francisco@] has joined #go-nuts
01:40 -!- franciscosouza_ [~francisco@] has joined #go-nuts
01:40 -!- hallas [~hallas@x1-6-30-46-9a-b2-c5-1f.k891.webspeed.dk] has quit [Ping
timeout: 276 seconds]
01:40 -!- franciscosouza [~francisco@] has quit [Read error:
Connection reset by peer]
01:48 -!- mehalelal [~mehalelal@] has joined #go-nuts
01:57 -!- _foocraft [~ewanas@] has joined #go-nuts
02:00 -!- Kami_ [~kami@unaffiliated/kami-/x-9078513] has quit [Excess Flood]
02:00 -!- exch [~blbl@ip34-181-209-87.adsl2.static.versatel.nl] has quit [Ping
timeout: 240 seconds]
02:00 -!- foocraft_ [~ewanas@] has quit [Ping timeout: 250 seconds]
02:01 -!- Kami_ [~kami@unaffiliated/kami-/x-9078513] has joined #go-nuts
02:02 -!- exch [~blbl@ip34-181-209-87.adsl2.static.versatel.nl] has joined
02:08 -!- Kami_ [~kami@unaffiliated/kami-/x-9078513] has quit [Quit: Off]
02:08 -!- Kami_ [~kami@unaffiliated/kami-/x-9078513] has joined #go-nuts
02:14 -!- nekoh [~nekoh@p4FC7007A.dip0.t-ipconnect.de] has quit [Quit: nekoh]
02:24 -!- _foocraft [~ewanas@] has quit [Ping timeout: 240 seconds]
02:29 -!- dj2 [~dj2@CPE001f5b35feb4-CM0014048e0344.cpe.net.cable.rogers.com] has
quit [Remote host closed the connection]
02:30 -!- dj2 [~dj2@CPE001f5b35feb4-CM0014048e0344.cpe.net.cable.rogers.com] has
joined #go-nuts
02:30 -!- angasule__ [~angasule@] has quit [Ping timeout: 240 seconds]
02:30 -!- wchicken [~chicken@c-98-248-194-46.hsd1.ca.comcast.net] has quit [Ping
timeout: 276 seconds]
02:35 -!- franciscosouza [~francisco@] has quit [Read error:
Connection reset by peer]
02:36 -!- franciscosouza [~francisco@] has joined #go-nuts
02:37 -!- dreadlorde [~dreadlord@c-24-11-39-160.hsd1.mi.comcast.net] has quit
[Ping timeout: 240 seconds]
02:37 -!- _foocraft [~ewanas@] has joined #go-nuts
02:43 -!- wchicken [~chicken@netblock-208-127-244-130.dslextreme.com] has joined
02:44 -!- keithcascio [~keithcasc@nat/google/x-dmrloerwtjglbfrc] has quit [Quit:
02:45 -!- jarsen [~jarsen@] has joined #go-nuts
02:47 -!- angasule [~angasule@] has joined #go-nuts
03:04 -!- niemeyer [~niemeyer@201-40-174-53.pltce701.dsl.brasiltelecom.net.br] has
quit [Ping timeout: 240 seconds]
03:09 -!- aat [~aat@cpe-72-225-174-173.nyc.res.rr.com] has joined #go-nuts
03:12 -!- jarsen [~jarsen@] has quit [Remote host closed the
03:12 -!- cafesofie [~cafesofie@ool-18b97779.dyn.optonline.net] has quit [Remote
host closed the connection]
03:17 -!- rputikar [~240182H@203-206-21-179.dyn.iinet.net.au] has joined #go-nuts
03:40 -!- angasule [~angasule@] has quit [Ping timeout: 244 seconds]
03:45 -!- rputikar [~240182H@203-206-21-179.dyn.iinet.net.au] has quit [Quit:
03:59 -!- wchicken [~chicken@netblock-208-127-244-130.dslextreme.com] has quit
[Ping timeout: 255 seconds]
04:06 -!- aaronblohowiak [47c606a4@gateway/web/freenode/ip.] has
joined #go-nuts
04:08 < aaronblohowiak> how would you create a map where the keys are
variable-length []byte's ?
04:11 -!- rejb [~rejb@unaffiliated/rejb] has quit [Disconnected by services]
04:11 -!- rejb [~rejb@unaffiliated/rejb] has joined #go-nuts
04:15 < KirkMcDonald> Slices can't be map keys.  Use strings instead?
04:16 -!- wchicken [~chicken@c-24-7-112-207.hsd1.ca.comcast.net] has joined
04:19 < aaronblohowiak> KirkMcDonald: is []byte -> string -> []byte
lossless for all values of []byte ? the #conversions doc makes me think so, but i
am cautious
04:20 < KirkMcDonald> It should be.  A string is a sequence of bytes.
04:20 < kevlar> aaronblohowiak: yes, though you don't want to do that
conversion often
04:20 < KirkMcDonald> Yes, it implies a copy.
04:20 < kevlar> you might want to hash your []byte and use that as the
string key and use the raw []byte as a part of the value
04:22 < aaronblohowiak> kevlar: that is a good idea, i could use a
complex128 to store a 16-byte hash of the key..
04:22 < KirkMcDonald> Though even given a key space as large as that, you're
boned if you get a collision.
04:23 < KirkMcDonald> However supremely unlikely that is.
04:26 < crunge> aaronblohowiak: what is it you're trying to accomplish/
04:26 < aaronblohowiak> crunge: the greatest yak shave the world has ever
04:26 < aaronblohowiak> crunge: i was thinking about writing a database with
an integrated mq that supports incremental map-reduce
04:27 < aaronblohowiak> s/database/redis-like thing/g
04:29 -!- franciscosouza [~francisco@] has quit [Read error:
Connection reset by peer]
04:29 -!- franciscosouza [~francisco@] has joined #go-nuts
04:30 -!- ExtraSpice [XtraSpice@78-57-204-104.static.zebra.lt] has joined #go-nuts
04:34 -!- boscop [~boscop@g226231009.adsl.alicedsl.de] has joined #go-nuts
04:34 -!- boscop [~boscop@g226231009.adsl.alicedsl.de] has quit [Changing host]
04:34 -!- boscop [~boscop@unaffiliated/boscop] has joined #go-nuts
04:36 -!- aat [~aat@cpe-72-225-174-173.nyc.res.rr.com] has quit [Quit: Computer
has gone to sleep.]
04:48 -!- Niedar [~bleh@ip68-99-166-222.hr.hr.cox.net] has quit [Read error:
Operation timed out]
04:49 -!- Niedar [~bleh@ip68-99-166-222.hr.hr.cox.net] has joined #go-nuts
04:56 -!- mehalelal [~mehalelal@] has quit [Quit: Leaving]
05:01 -!- fabled [~fabled@] has joined #go-nuts
05:01 -!- zozoR [~Morten@2906ds2-arno.0.fullrate.dk] has joined #go-nuts
05:02 -!- Tv [~Tv@cpe-76-168-227-45.socal.res.rr.com] has quit [Ping timeout: 240
05:04 -!- va3atc [~va3atc@24-246-17-37.cable.teksavvy.com] has quit []
05:21 -!- aaronblohowiak [47c606a4@gateway/web/freenode/ip.] has quit
[Quit: Page closed]
05:24 -!- Cobi [~Cobi@2002:1828:88fb:0:aede:48ff:febe:ef03] has quit [Ping
timeout: 260 seconds]
05:26 -!- Stiletto [7f000001@] has quit [Ping timeout: 260 seconds]
05:31 -!- Stiletto [7f000001@] has joined #go-nuts
05:37 -!- Cobi [~Cobi@2002:1828:88fb:0:aede:48ff:febe:ef03] has joined #go-nuts
05:50 -!- NiteRain [~kvirc@c-98-254-236-21.hsd1.fl.comcast.net] has quit [Read
error: Operation timed out]
05:54 -!- moraes [~moraes@] has quit [Ping timeout: 276 seconds]
05:56 -!- alehorst [~alehorst@] has quit [Quit: Leaving.]
05:58 -!- moraes [~moraes@] has joined #go-nuts
05:59 -!- _foocraft [~ewanas@] has quit [Ping timeout: 255 seconds]
06:05 -!- iant [~iant@] has quit [Ping timeout: 240 seconds]
06:22 -!- iant [~iant@adsl-71-133-8-30.dsl.pltn13.pacbell.net] has joined #go-nuts
06:23 -!- mode/#go-nuts [+v iant] by ChanServ
06:25 -!- noodles775 [~michael@g225134255.adsl.alicedsl.de] has joined #go-nuts
06:25 -!- noodles775 [~michael@g225134255.adsl.alicedsl.de] has quit [Changing
06:25 -!- noodles775 [~michael@canonical/launchpad/noodles775] has joined #go-nuts
06:25 -!- bortzmeyer [~bortzmeye@batilda.nic.fr] has joined #go-nuts
06:31 -!- nannto [~nanto@pee5b70.tokyff01.ap.so-net.ne.jp] has quit [Read error:
Connection reset by peer]
06:32 -!- nannto [~nanto@pee5b70.tokyff01.ap.so-net.ne.jp] has joined #go-nuts
06:35 -!- thomas_b [~thomasb@cm-] has joined #go-nuts
06:47 -!- photron [~photron@port-92-201-145-30.dynamic.qsc.de] has joined #go-nuts
06:48 -!- wrtp [~rog@] has joined #go-nuts
06:49 -!- wrtp [~rog@] has quit [Client Quit]
06:49 -!- sebastianskejoe [~sebastian@] has joined #go-nuts
06:54 -!- coderendezvous [~coderende@adsl-98-66-2-41.mem.bellsouth.net] has joined
07:00 -!- Guest39274 [~piranha@a62-251-16-146.adsl.xs4all.nl] has joined #go-nuts
07:00 -!- jemeshsu [~jemeshsu@bb220-255-88-127.singnet.com.sg] has quit [Quit:
07:02 -!- bartbes [~bartbes@love/developer/bartbes] has joined #go-nuts
07:08 -!- ptrb [~peter@archimedes.bourgon.org] has quit [Ping timeout: 240
07:10 -!- ptrb [~peter@archimedes.bourgon.org] has joined #go-nuts
07:11 -!- chaos95 [chaos95@mafianode.com] has quit [Ping timeout: 276 seconds]
07:11 -!- Guest39274 [~piranha@a62-251-16-146.adsl.xs4all.nl] has quit [Ping
timeout: 246 seconds]
07:12 -!- chaos95 [chaos95@mafianode.com] has joined #go-nuts
07:15 -!- coderendezvous [~coderende@adsl-98-66-2-41.mem.bellsouth.net] has quit
[Quit: This computer has gone to sleep]
07:16 -!- b33p [~mgray@li226-224.members.linode.com] has joined #go-nuts
07:16 -!- ptrb [~peter@archimedes.bourgon.org] has quit [Remote host closed the
07:17 -!- coderendezvous [~coderende@adsl-98-66-2-41.mem.bellsouth.net] has joined
07:17 -!- ptrb [~peter@archimedes.bourgon.org] has joined #go-nuts
07:19 -!- chaos95 [chaos95@mafianode.com] has quit [Ping timeout: 246 seconds]
07:19 -!- piranha_ [~piranha@a62-251-16-146.adsl.xs4all.nl] has joined #go-nuts
07:21 -!- virtualsue [~chatzilla@nat/cisco/x-uegelxzufdlkudwu] has joined #go-nuts
07:21 -!- napsy [~luka@] has joined #go-nuts
07:26 -!- muke [~doobies@75-59-237-124.lightspeed.sntcca.sbcglobal.net] has quit
[Read error: Connection reset by peer]
07:26 -!- muke [~doobies@75-59-237-124.lightspeed.sntcca.sbcglobal.net] has joined
07:28 -!- ptrb [~peter@archimedes.bourgon.org] has quit [Ping timeout: 260
07:28 -!- chaos95 [chaos95@mafianode.com] has joined #go-nuts
07:28 -!- ptrb [~peter@archimedes.bourgon.org] has joined #go-nuts
07:31 -!- Ekspluati [3e4ed8ee@gateway/web/freenode/ip.] has joined
07:31 -!- franciscosouza_ [~francisco@] has joined #go-nuts
07:31 -!- franciscosouza [~francisco@] has quit [Read error:
Connection reset by peer]
07:32 -!- Project_2501 [~Marvin@dynamic-adsl-94-36-176-214.clienti.tiscali.it] has
joined #go-nuts
07:32 -!- mikespook [~mikespook@] has quit [Quit: Leaving.]
07:32 -!- mikespook [~mikespook@] has joined #go-nuts
07:38 -!- neshaug [~oyvind@] has joined #go-nuts
07:40 -!- neshaug [~oyvind@] has quit [Client Quit]
07:40 -!- dfc [~dfc@eth59-167-133-99.static.internode.on.net] has quit [Ping
timeout: 246 seconds]
07:40 -!- neshaug [~oyvind@] has joined #go-nuts
07:49 -!- |Craig| [~|Craig|@panda3d/entropy] has quit [Quit: |Craig|]
07:55 -!- wrtp [~rog@] has joined #go-nuts
07:55 -!- wrtp [~rog@] has left #go-nuts []
08:00 -!- GeertJohan [~Squarc@clal-1-229.eduroam.inholland.nl] has joined #go-nuts
08:10 -!- coderendezvous [~coderende@adsl-98-66-2-41.mem.bellsouth.net] has quit
[Quit: Leaving]
08:15 -!- Fish [~Fish@exo3753.pck.nerim.net] has quit [Quit: So Long, and Thanks
for All the Fish]
08:18 -!- Fish [~Fish@exo3753.pck.nerim.net] has joined #go-nuts
08:23 -!- sahid [~sahid@LNeuilly-152-21-22-10.w193-253.abo.wanadoo.fr] has joined
08:26 -!- Project_2501 [~Marvin@dynamic-adsl-94-36-176-214.clienti.tiscali.it] has
quit [Read error: Connection reset by peer]
08:28 -!- Project_2501 [~Marvin@dynamic-adsl-94-36-153-209.clienti.tiscali.it] has
joined #go-nuts
08:30 -!- GeertJohan [~Squarc@clal-1-229.eduroam.inholland.nl] has quit [Ping
timeout: 276 seconds]
08:31 -!- bortzmeyer [~bortzmeye@batilda.nic.fr] has quit [Ping timeout: 260
08:31 -!- dfc [~dfc@124-149-49-45.dyn.iinet.net.au] has joined #go-nuts
08:34 -!- GeertJohan [~Squarc@clal-1-229.eduroam.inholland.nl] has joined #go-nuts
08:40 -!- bobody [~alexc@unaffiliated/alexc] has joined #go-nuts
08:40 -!- GeertJohan [~Squarc@clal-1-229.eduroam.inholland.nl] has quit [Read
error: Connection reset by peer]
08:42 -!- zozoR [~Morten@2906ds2-arno.0.fullrate.dk] has quit [Ping timeout: 255
08:52 -!- jstemmer [~cheetah@mrpwn.stemmertech.com] has joined #go-nuts
08:59 < exch> 80's cartoon themesongs are great productivity enhancers.
Specially the cheesy hairmetal ones \o/
08:59 -!- franciscosouza [~francisco@] has quit [Read error:
Operation timed out]
09:09 -!- marko_ [~marko@host213-200-dynamic.36-79-r.retail.telecomitalia.it] has
joined #go-nuts
09:20 -!- bartbes [~bartbes@love/developer/bartbes] has quit [Ping timeout: 250
09:21 -!- arun [~arun@unaffiliated/sindian] has quit [Read error: Connection reset
by peer]
09:23 -!- arun [~arun@2001:610:110:4e2:280:5aff:fe69:e130] has joined #go-nuts
09:23 -!- arun [~arun@2001:610:110:4e2:280:5aff:fe69:e130] has quit [Changing
09:23 -!- arun [~arun@unaffiliated/sindian] has joined #go-nuts
09:31 -!- jlouis [jlouis@horus.0x90.dk] has quit [Ping timeout: 240 seconds]
09:31 -!- jlouis [jlouis@horus.0x90.dk] has joined #go-nuts
09:38 -!- Sisten [~Sisten@s213-103-208-147.cust.tele2.se] has joined #go-nuts
09:41 -!- sacho [~sacho@87-126-53-242.btc-net.bg] has quit [Ping timeout: 248
09:42 -!- sacho [~sacho@87-126-53-242.btc-net.bg] has joined #go-nuts
09:44 -!- bobody [~alexc@unaffiliated/alexc] has quit [Quit: WeeChat 0.3.4]
09:45 -!- rputikar [~240182H@203-206-21-179.dyn.iinet.net.au] has joined #go-nuts
09:57 -!- sacho [~sacho@87-126-53-242.btc-net.bg] has quit [Ping timeout: 248
09:58 -!- tvw [~tv@] has joined #go-nuts
09:59 -!- zcram [~zcram@78-28-65-177.cdma.dyn.kou.ee] has joined #go-nuts
10:00 -!- zcram [~zcram@78-28-65-177.cdma.dyn.kou.ee] has quit [Client Quit]
10:01 -!- coffeejunk [~max@static.] has quit
[Remote host closed the connection]
10:02 -!- Kami_ [~kami@unaffiliated/kami-/x-9078513] has quit [Excess Flood]
10:04 -!- Kami_ [~kami@unaffiliated/kami-/x-9078513] has joined #go-nuts
10:10 -!- Count_Niedar [~bleh@ip68-99-166-222.hr.hr.cox.net] has joined #go-nuts
10:12 -!- Sebastian_ [~Sebastian@git.sebastianhahn.net] has joined #go-nuts
10:13 -!- Niedar [~bleh@ip68-99-166-222.hr.hr.cox.net] has quit [Ping timeout: 248
10:13 -!- bobody [~alexc@unaffiliated/alexc] has joined #go-nuts
10:14 < Sebastian_> Hey there, a while ago I asked about the development of
the crypto package (see
).  Is anyone available to provide more insight in what might be next, and how
mature the crypto package is thought to be generally?
10:14 -!- Ekspluati [3e4ed8ee@gateway/web/freenode/ip.] has quit
[Ping timeout: 252 seconds]
10:16 -!- wrtp [~rog@] has joined #go-nuts
10:25 -!- shvntr [~shvntr@] has joined #go-nuts
10:27 -!- ronnyy [~quassel@p4FF1C439.dip0.t-ipconnect.de] has joined #go-nuts
10:32 < xyproto> How can I know a good size for the buffer when reading from
a file?  4000 bytes?
10:33 < xyproto> Is it silly to adjust the buffer size during reading, for
optimal speed?
10:33 < xyproto> (really large files)
10:40 < dfc> xyproto: just benchmark it once
10:40 < dfc> then use that value
10:41 < dfc> most common benchmarks show 90% of the benefit is gained with
10:41 < dfc> 98% with 65536
10:42 -!- nannto [~nanto@pee5b70.tokyff01.ap.so-net.ne.jp] has quit [Quit:
10:45 < wrtp> i usually use 8192
10:46 < wrtp> if know i'm reading a really big file (e.g.  copying a whole
disk) then i sometimes up it to 1024*1024
10:46 < wrtp> it also helps to overlap IO
10:47 < wrtp> interestingly the default buffer size for bufio is 4096
10:47 -!- bartbes [~bartbes@love/developer/bartbes] has joined #go-nuts
10:48 < wrtp> i think that's because of the memory trade-off - bufio is
often used for other features in addition to its buffering (e.g.  rune, line
10:49 < wrtp> xyproto: adjusting buffer size during reading would add
complexity for no discernible benefit IMHO
10:49 -!- rlab [~Miranda@] has joined #go-nuts
10:49 < xyproto> wrtp: I see.  No recommendations of particle swarm
optimizations, then ;)
10:49 < xyproto> dfc: thanks for the size suggestion
10:50 < wrtp> xyproto: not when the benefits are so small...
10:50 < xyproto> I'll try both 8192 and 65536, then
10:54 < wrtp> xyproto: be careful to watch out for cache effects...
10:57 -!- alehorst [~alehorst@] has joined #go-nuts
10:59 -!- exch [~blbl@ip34-181-209-87.adsl2.static.versatel.nl] has quit [Ping
timeout: 244 seconds]
11:01 -!- angasule [~angasule@] has joined #go-nuts
11:04 -!- prakti [~travis@Brain.cs.uni-paderborn.de] has joined #go-nuts
11:07 < dfc> xyproto: you should do some googling, and some benchmarking
11:08 < dfc> for applications i've written, which are mostly network based,
65536 has been more than enough
11:12 -!- virtualsue [~chatzilla@nat/cisco/x-uegelxzufdlkudwu] has quit [Remote
host closed the connection]
11:12 -!- franciscosouza [~francisco@] has joined #go-nuts
11:24 < skelterjohn> xyproto: consider a slice of slices
11:26 -!- goon12 [~goon12@71-87-215-29.dhcp.oxfr.ma.charter.com] has quit [Ping
timeout: 240 seconds]
11:27 -!- goon12 [~goon12@71-87-215-29.dhcp.oxfr.ma.charter.com] has joined
11:27 -!- LeNsTR [~lenstr@1.qs.biz] has quit [Changing host]
11:27 -!- LeNsTR [~lenstr@unaffiliated/lenstr] has joined #go-nuts
11:28 -!- tncardoso [~thiago@] has joined #go-nuts
11:31 -!- nekoh [~nekoh@p4FC706AE.dip0.t-ipconnect.de] has joined #go-nuts
11:35 -!- exch [~blbl@] has joined #go-nuts
11:35 -!- coffeejunk [~max@static.] has joined
11:38 -!- franciscosouza_ [~francisco@] has joined #go-nuts
11:38 -!- franciscosouza [~francisco@] has quit [Read error:
Connection reset by peer]
11:41 -!- ronnyy [~quassel@p4FF1C439.dip0.t-ipconnect.de] has quit [Remote host
closed the connection]
11:53 -!- jyxent [~jyxent@] has quit [Ping timeout: 244 seconds]
11:55 -!- jyxent [~jyxent@] has joined #go-nuts
12:01 -!- franciscosouza [~francisco@] has quit [Quit:
12:03 -!- firwen [~firwen@adevlaptop.cern.ch] has joined #go-nuts
12:03 -!- firwen [~firwen@adevlaptop.cern.ch] has quit [Client Quit]
12:04 -!- piranha [~piranha@a62-251-16-146.adsl.xs4all.nl] has quit [Ping timeout:
240 seconds]
12:04 -!- piranha [~piranha@D57D1AB3.static.ziggozakelijk.nl] has joined #go-nuts
12:08 -!- zozoR [~Morten@2906ds2-arno.0.fullrate.dk] has joined #go-nuts
12:10 -!- ronnyy [~quassel@p4FF1C439.dip0.t-ipconnect.de] has joined #go-nuts
12:11 -!- piranha [~piranha@D57D1AB3.static.ziggozakelijk.nl] has quit [Ping
timeout: 240 seconds]
12:17 -!- bobody [~alexc@unaffiliated/alexc] has quit [Quit: WeeChat 0.3.4]
12:19 -!- niemeyer [~niemeyer@201-40-174-53.pltce701.dsl.brasiltelecom.net.br] has
joined #go-nuts
12:26 -!- ronnyy [~quassel@p4FF1C439.dip0.t-ipconnect.de] has quit [Remote host
closed the connection]
12:28 -!- royger [~royger@seldon.upc.es] has joined #go-nuts
12:28 < royger> hello
12:29 < royger> does someone have an example of an http GET in Go using the
12:33 -!- th0re [~thre@p5B3B5491.dip.t-dialin.net] has joined #go-nuts
12:40 -!- wrtp [~rog@] has quit [Quit: wrtp]
12:42 -!- wrtp [~rog@] has joined #go-nuts
12:45 -!- wrtp [~rog@] has quit [Client Quit]
12:45 -!- angasule [~angasule@] has quit [Ping timeout: 250 seconds]
12:47 -!- th0re [~thre@p5B3B5491.dip.t-dialin.net] has quit [Quit: Der weg zur
erkenntniss ist der richtige.]
12:50 -!- franciscosouza [~francisco@] has joined #go-nuts
12:54 -!- replore [~replore@ntkngw256114.kngw.nt.ftth.ppp.infoweb.ne.jp] has
joined #go-nuts
12:54 -!- adlan [~adlan@] has joined #go-nuts
12:57 -!- dlowe [~dlowe@ita4fw1.itasoftware.com] has joined #go-nuts
13:00 -!- r_linux [~r_linux@static.] has joined
13:04 -!- franciscosouza [~francisco@] has quit [Quit: franciscosouza]
13:07 -!- franciscosouza [~francisco@] has joined #go-nuts
13:07 -!- noam [noam@] has quit [Ping timeout: 240
13:07 -!- gtaylor [~gtaylor@99-126-136-139.lightspeed.gnvlsc.sbcglobal.net] has
joined #go-nuts
13:08 -!- sebastianskejoe [~sebastian@] has quit [Quit: Lost
13:09 -!- franciscosouza [~francisco@] has quit [Client Quit]
13:11 -!- noam [noam@] has joined #go-nuts
13:16 -!- franciscosouza [~francisco@] has joined #go-nuts
13:18 -!- franciscosouza [~francisco@] has quit [Client Quit]
13:26 -!- iant [~iant@adsl-71-133-8-30.dsl.pltn13.pacbell.net] has quit [Quit:
13:29 -!- dj2 [~dj2@CPE001f5b35feb4-CM0014048e0344.cpe.net.cable.rogers.com] has
quit [Remote host closed the connection]
13:35 -!- purplegrape [~baalsoffi@cm133.gamma204.maxonline.com.sg] has joined
13:38 -!- iant [~iant@] has joined #go-nuts
13:38 -!- mode/#go-nuts [+v iant] by ChanServ
13:41 -!- piranha [~piranha@D57D1AB3.static.ziggozakelijk.nl] has joined #go-nuts
13:45 -!- napsy [~luka@] has quit [Ping timeout: 276 seconds]
13:48 -!- [muttox] [~dheppell@] has quit [Remote
host closed the connection]
13:50 -!- dj2 [~dj2@] has joined #go-nuts
13:50 -!- franciscosouza [~francisco@] has joined #go-nuts
13:50 -!- zozoR [~Morten@2906ds2-arno.0.fullrate.dk] has quit [Ping timeout: 240
13:51 -!- chomp [~chomp@dap-209-166-184-50.pri.tnt-3.pgh.pa.stargate.net] has
joined #go-nuts
13:52 -!- [muttox] [~dheppell@] has joined
13:53 -!- Fish [~Fish@exo3753.pck.nerim.net] has quit [Quit: So Long, and Thanks
for All the Fish]
13:54 -!- napsy [~luka@] has joined #go-nuts
13:54 -!- moraes_ [~moraes@] has joined #go-nuts
13:55 -!- nsf [~nsf@jiss.convex.ru] has quit [Quit: WeeChat 0.3.5]
13:56 -!- moraes [~moraes@] has quit [Ping timeout: 260 seconds]
13:57 -!- Fish [~Fish@exo3753.pck.nerim.net] has joined #go-nuts
13:57 -!- aat [~aat@rrcs-184-75-54-130.nyc.biz.rr.com] has joined #go-nuts
14:01 -!- GeertJohan [~Squarc@] has joined #go-nuts
14:03 -!- royger [~royger@seldon.upc.es] has left #go-nuts []
14:11 -!- nekoh [~nekoh@p4FC706AE.dip0.t-ipconnect.de] has quit [Ping timeout: 240
14:11 -!- xyproto [~alexander@] has quit [Quit: WeeChat 0.3.5]
14:27 -!- xulfer [~xulfer@2600:3c01::f03c:91ff:fe96:c1cd] has joined #go-nuts
14:27 -!- adlan [~adlan@] has quit [Remote host closed the
14:34 -!- Adys [~Adys@unaffiliated/adys] has quit [Read error: Connection reset by
14:39 -!- GeertJohan [~Squarc@] has quit [Quit: Leaving.]
14:42 -!- jbooth1 [~jay@] has joined #go-nuts
14:42 -!- wrtp [~rog@] has joined #go-nuts
14:45 -!- franciscosouza [~francisco@] has quit [Quit: franciscosouza]
14:45 -!- rlab [~Miranda@] has quit [Ping timeout: 240 seconds]
14:48 -!- Project-2501 [~Marvin@] has joined #go-nuts
14:48 -!- Adys [~Adys@unaffiliated/adys] has joined #go-nuts
14:50 -!- Project_2501 [~Marvin@dynamic-adsl-94-36-153-209.clienti.tiscali.it] has
quit [Ping timeout: 240 seconds]
14:54 -!- tncardoso [~thiago@] has quit [Read error: Connection
reset by peer]
14:56 -!- pharris [~Adium@rhgw.opentext.com] has joined #go-nuts
14:58 -!- dreadlorde [~dreadlord@c-24-11-39-160.hsd1.mi.comcast.net] has joined
15:00 -!- espeed [~espeed@] has quit [Quit: Leaving]
15:01 -!- r_linux [~r_linux@static.] has quit
[Ping timeout: 240 seconds]
15:01 -!- r_linux [~r_linux@static.] has joined
15:01 -!- iant [~iant@] has quit [Quit: Leaving.]
15:08 -!- shvntr [~shvntr@] has quit [Ping timeout: 260 seconds]
15:10 -!- virtualsue [~chatzilla@nat/cisco/x-ebzybyvupjsqgzje] has joined #go-nuts
15:10 -!- tncardoso [~thiago@] has joined #go-nuts
15:14 -!- r_linux [~r_linux@static.] has quit
[Ping timeout: 250 seconds]
15:15 -!- r_linux [~r_linux@static.] has joined
15:16 -!- zcram [~zcram@78-28-88-7.cdma.dyn.kou.ee] has joined #go-nuts
15:17 -!- nekoh [~nekoh@p4FC70478.dip0.t-ipconnect.de] has joined #go-nuts
15:21 -!- purplegrape [~baalsoffi@cm133.gamma204.maxonline.com.sg] has quit []
15:22 -!- xyproto [~alexander@] has joined #go-nuts
15:27 -!- homa_rano [~ede@30-51-251.dynamic.csail.mit.edu] has joined #go-nuts
15:28 -!- Ekspluati [3e4ed8ee@gateway/web/freenode/ip.] has joined
15:28 -!- nekoh [~nekoh@p4FC70478.dip0.t-ipconnect.de] has quit [Ping timeout: 255
15:33 -!- franciscosouza [~francisco@] has joined #go-nuts
15:34 -!- Ekspluati [3e4ed8ee@gateway/web/freenode/ip.] has quit
[Ping timeout: 252 seconds]
15:36 -!- rlab [~Miranda@] has joined #go-nuts
15:37 -!- eimantas [~eimantas@ip-212-52-52-163.kava.lt] has joined #go-nuts
15:37 -!- Ekspluati [3e4ed8ee@gateway/web/freenode/ip.] has joined
15:38 -!- moraes_ [~moraes@] has quit [Remote host closed the
15:40 -!- dlowe [~dlowe@ita4fw1.itasoftware.com] has quit [Quit: Leaving.]
15:40 -!- noodles775 [~michael@canonical/launchpad/noodles775] has quit [Quit:
15:42 -!- GeertJohan [~Squarc@ip4da06866.direct-adsl.nl] has joined #go-nuts
15:45 -!- alehorst [~alehorst@] has quit [Ping timeout: 255
15:46 -!- xyproto [~alexander@] has quit [Read error: No route to
15:48 -!- tncardoso [~thiago@] has quit [Quit: bye]
15:48 -!- kanliot-noob [~chatzilla@] has joined #go-nuts
15:52 -!- xyproto [~alexander@] has joined #go-nuts
15:52 -!- chomp [~chomp@dap-209-166-184-50.pri.tnt-3.pgh.pa.stargate.net] has quit
[Quit: Leaving]
15:54 -!- ijknacho [48be4003@gateway/web/freenode/ip.] has joined
15:56 -!- Venom_X [~pjacobs@75-27-133-72.lightspeed.austtx.sbcglobal.net] has
joined #go-nuts
15:56 -!- pjacobs [~pjacobs@75-27-133-72.lightspeed.austtx.sbcglobal.net] has
joined #go-nuts
15:56 -!- pjacobs [~pjacobs@75-27-133-72.lightspeed.austtx.sbcglobal.net] has quit
[Read error: Connection reset by peer]
15:58 -!- alehorst [~alehorst@] has joined
15:58 -!- coffeejunk [~max@static.] has quit
[Quit: oh my.]
16:01 -!- rputikar [~240182H@203-206-21-179.dyn.iinet.net.au] has quit [Quit:
16:01 -!- piranha [~piranha@D57D1AB3.static.ziggozakelijk.nl] has quit [Quit:
Computer has gone to sleep.]
16:02 -!- nekoh [~nekoh@p4FC70335.dip0.t-ipconnect.de] has joined #go-nuts
16:03 -!- coffeejunk [~max@static.] has joined
16:05 -!- alehorst [~alehorst@] has quit [Ping
timeout: 255 seconds]
16:10 -!- wrtp [~rog@] has quit [Quit: wrtp]
16:10 -!- zcram [~zcram@78-28-88-7.cdma.dyn.kou.ee] has quit [Remote host closed
the connection]
16:17 -!- alehorst [~alehorst@] has joined #go-nuts
16:17 -!- nekoh [~nekoh@p4FC70335.dip0.t-ipconnect.de] has quit [Ping timeout: 255
16:20 -!- tncardoso [~thiagon@] has joined #go-nuts
16:26 -!- zozoR [~Morten@2906ds2-arno.0.fullrate.dk] has joined #go-nuts
16:26 -!- zozoR [~Morten@2906ds2-arno.0.fullrate.dk] has quit [Read error:
Connection reset by peer]
16:27 -!- zozoR [~Morten@2906ds2-arno.0.fullrate.dk] has joined #go-nuts
16:27 -!- virtualsue [~chatzilla@nat/cisco/x-ebzybyvupjsqgzje] has quit [Ping
timeout: 244 seconds]
16:27 -!- firwen [~firwen@2a01:e34:eea3:7e10:4a5b:39ff:fe51:e8ae] has joined
16:28 -!- dreadlorde [~dreadlord@c-24-11-39-160.hsd1.mi.comcast.net] has quit
[Ping timeout: 276 seconds]
16:36 -!- prakti [~travis@Brain.cs.uni-paderborn.de] has quit [Quit: Hackedi
hackedi weg]
16:37 -!- iant [~iant@nat/google/x-mtmodrzjenewzgms] has joined #go-nuts
16:37 -!- mode/#go-nuts [+v iant] by ChanServ
16:38 -!- chomp [~chomp@dap-209-166-184-50.pri.tnt-3.pgh.pa.stargate.net] has
joined #go-nuts
16:39 -!- ronnyy [~quassel@p4FF1C439.dip0.t-ipconnect.de] has joined #go-nuts
16:39 -!- Tv [~Tv@ip-66-33-206-8.dreamhost.com] has joined #go-nuts
16:40 -!- napsy [~luka@] has quit [Quit: Lost terminal]
16:43 -!- TheMue [~TheMue@p5DDF52EC.dip.t-dialin.net] has joined #go-nuts
16:43 -!- GeertJohan [~Squarc@ip4da06866.direct-adsl.nl] has quit [Ping timeout:
250 seconds]
16:49 -!- brett [~brett@rdnzl.net] has left #go-nuts []
16:49 -!- drhodes [~none@drhodes.xen.prgmr.com] has joined #go-nuts
16:57 -!- tobym [~tobym@rrcs-208-105-11-82.nyc.biz.rr.com] has joined #go-nuts
17:04 < kanliot-noob> i managed to install go-gtk, but I had to do a "hg
install tips" someone explain to me what i just did?  http://pastie.org/2033195
17:07 < skelterjohn|work> huh - no idea what hg install does
17:07 < kanliot-noob> yeah i was googling it and got nowhere
17:10 -!- richard_iii [~richard_i@] has joined #go-nuts
17:11 < yebyen> is there anything you have to do when using fmt.Print to
flush stdout?
17:12 < aiju> fmt.Print is unbuffered afaik
17:12 < yebyen> so then, i'm going to pastebin this code...
17:12 < yebyen> since i can't seem to determine what's going wrong
17:12 < yebyen> http://pastebin.com/pPvyTdKZ
17:13 < yebyen> ignore the call to os.Stdout.Flush()
17:13 < yebyen> since it doesn't exist...
17:13 -!- ijknacho [48be4003@gateway/web/freenode/ip.] has quit [Ping
timeout: 252 seconds]
17:14 < yebyen> hmm
17:14 < yebyen> never mind, gb is not copying my binaries to the right place
17:14 < yebyen> i'm running an old copy with no print statements :(
17:15 < yebyen> better!  i'm running /usr/bin/test, instead of my intended
17:16 < skelterjohn|work> what's gb not doing right?
17:16 < skelterjohn|work> and what i mean to say is, what do you think the
"right" place is?
17:16 < yebyen> into my path
17:16 < yebyen> don't worry, your tool is functioning correctly
17:16 < yebyen> there is another "test" earlier in my path
17:17 < yebyen> /usr/bin/test
17:17 -!- tobym [~tobym@rrcs-208-105-11-82.nyc.biz.rr.com] has quit [Ping timeout:
276 seconds]
17:17 < skelterjohn|work> test is a tricky name
17:17 < skelterjohn|work> if it's just a test binary, don't install it
17:17 < skelterjohn|work> run gb and then ./bin/test
17:17 < yebyen> i'm writing a translator from spreadsheets into amazon data
17:17 < yebyen> today
17:18 < yebyen> should be fun :(
17:18 < yebyen> skelterjohn|work: i don't mind polluting $HOME/bin, i'll
clean it out later...
17:18 < yebyen> i thought it was a weird issue with running something not in
the current directory
17:18 < yebyen> so i installed it
17:18 < skelterjohn|work> ok, as long as you know that when you don't -i,
the binaries are still accessible
17:18 < skelterjohn|work> gotcha
17:19 < yebyen> since they seem to inherit their working directory from
their own location, and not the current path, if you invoke them like that...
17:19 < yebyen> i have seen that before anyway
17:19 < skelterjohn|work> gb makes it easy to clean out, too - the -N option
is nuke.
17:19 < skelterjohn|work> but that will remove whatever file exists where gb
wants to install
17:19 < yebyen> yeah
17:19 < yebyen> danger will robinson
17:19 < skelterjohn|work> (so be careful)
17:20 < skelterjohn|work> it asks for confirmations, too
17:22 -!- KingPhilroy [~kingphilr@68-116-192-86.dhcp.oxfr.ma.charter.com] has quit
[Quit: Sleeping with the fishes...]
17:23 -!- KingPhilroy [~kingphilr@68-116-192-86.dhcp.oxfr.ma.charter.com] has
joined #go-nuts
17:25 -!- franciscosouza [~francisco@] has quit [Quit: franciscosouza]
17:26 -!- KingPhilroy [~kingphilr@68-116-192-86.dhcp.oxfr.ma.charter.com] has quit
[Remote host closed the connection]
17:26 -!- firwen [~firwen@2a01:e34:eea3:7e10:4a5b:39ff:fe51:e8ae] has quit [Remote
host closed the connection]
17:33 -!- franciscosouza [~francisco@] has joined #go-nuts
17:37 < skelterjohn|work> i wish someone made a nice 2d geometry library
17:37 < skelterjohn|work> now i have to make one
17:40 < yebyen> Go-SDL is no good?
17:41 < KirkMcDonald> That's not a geometry library as such, is it?
17:41 < chomp> not exactly a 2d geometry library
17:41 < yebyen> ah...  not exactly a library for drawing charts, either
17:42 < skelterjohn|work> i mean something for manipulating and reasoning
about geometric shapes
17:43 < chomp> for computational or visualization purposes?  or both?
17:43 < skelterjohn|work> first one, then the other
17:43 < chomp> ah.
17:43 < skelterjohn|work> but in my case i only need the first
17:43 -!- firwen [~firwen@2a01:e34:eea3:7e10:4a5b:39ff:fe51:e8ae] has joined
17:44 -!- moraes [~moraes@] has joined #go-nuts
17:45 < chomp> well it's really only tangential and certainly of no use in
that regard, but it reminded me of this old piece:
17:46 < skelterjohn|work> unsigned int b = I---------I;
17:46 < skelterjohn|work> !!!!!!!!!
17:46 < aiju> have you added dongml support to C now?
17:47 < skelterjohn|work> that is amazing
17:47 < chomp> heh i'm a fan of the cuboid
17:51 -!- GeertJohan [~Squarc@ip4da06866.direct-adsl.nl] has joined #go-nuts
17:55 -!- GeertJohan [~Squarc@ip4da06866.direct-adsl.nl] has quit [Read error:
Connection reset by peer]
17:55 -!- GeertJohan [~Squarc@ip4da06866.direct-adsl.nl] has joined #go-nuts
18:04 -!- Fish- [~Fish@9fans.fr] has joined #go-nuts
18:08 -!- franciscosouza [~francisco@] has quit [Quit: franciscosouza]
18:09 -!- kanliot-noob [~chatzilla@] has quit [Remote host closed the
18:18 -!- robteix [~robteix@] has joined #go-nuts
18:29 -!- tncardoso [~thiagon@] has quit [Quit: Leaving]
18:34 -!- rutkowski [~adrian@078088210082.walbrzych.vectranet.pl] has joined
18:34 < askhader> When I perform an http get request with http.get(url), if
the url does not resolve correctly the following runtime error is encountered.  Is
this the expected behaviour?  http://pastebin.com/A7SEubb7
18:37 -!- tvw [~tv@] has quit [Remote host closed the connection]
18:37 < exch> It shouldn't panic but is difficult to say without seeing some
code..  it fails in 'scrapeOMGUW/go/fetch.go:12'..  what's in that function?
18:39 < skelterjohn|work> askhader: often when a problem is described like
that the issue is the code not checking errors
18:39 < skelterjohn|work> do you check error returns?  and print them out if
they aren't nil?
18:40 < askhader> This is the actual .get() call that is erring
18:40 -!- kr [~Keith@] has joined #go-nuts
18:40 < askhader> Now .get() returns an error, but I don't know of a way to
check for it before I get the call.
18:40 < skelterjohn|work> ?
18:40 < askhader> Aha!
18:40 < skelterjohn|work> i don't understand
18:41 < askhader> No need, I've figured it.
18:41 < skelterjohn|work> how do you know it returns an error if there is no
way to check for it
18:41 < askhader> Error checking was my problem.
18:41 < skelterjohn|work> cool
18:41 < askhader> I apologize, rather new to Go.
18:41 < exch> can you pastebin a few lines surrounding the offending bit?
That should at least give us a point to start from
18:43 -!- franciscosouza [~francisco@] has joined #go-nuts
18:44 -!- |Craig| [~|Craig|@panda3d/entropy] has joined #go-nuts
18:54 < GeertJohan> it seems gob is not able to export complex
datastrucures..  I have this struct: type Player struct { Username string;
Password string; reference *PlayerReference; Inventory Inventory; lock
*sync.RWMutex };; type Inventory struct {Items list.List; Lock sync.RWMutex }.
When encoding a Player instance I get an error: "gob: type list.List has no
exported fields"
18:54 -!- foocraft [~ewanas@] has joined #go-nuts
18:55 < GeertJohan> so, can gob do this and am I doing something wrong?
18:55 -!- tobym [~tobym@rrcs-208-105-11-82.nyc.biz.rr.com] has joined #go-nuts
18:55 < fvbommel> gob (like any other package) can't access private fields
of other packages.
18:56 < fvbommel> So if list.List has private fields, it can't touch them.
18:56 < GeertJohan> ok
18:57 -!- r_linux [~r_linux@static.] has quit
[Ping timeout: 240 seconds]
18:57 -!- replore [~replore@ntkngw256114.kngw.nt.ftth.ppp.infoweb.ne.jp] has quit
[Remote host closed the connection]
18:58 < fvbommel> If you need to transmit such types you may be interested
in http://golang.org/pkg/gob/#GobEncoder and
18:58 < fvbommel> Alternatively, you could maybe use a []Item instead of a
19:03 < GeertJohan> probably yes, though a []item is not dynamicly
19:03 -!- tobym [~tobym@rrcs-208-105-11-82.nyc.biz.rr.com] has quit [Ping timeout:
258 seconds]
19:03 < GeertJohan> so I should make a tree myself I guess.
19:03 < GeertJohan> anyway, thx :)
19:04 < skelterjohn|work> []item is dynamically expandable
19:04 < GeertJohan> nah wait
19:04 < GeertJohan> it is :o ?
19:04 < skelterjohn|work> theItemSlice = append(theItemSlice, aNewItem)
19:04 < GeertJohan> :O oh :P X
19:04 < GeertJohan> XD
19:04 -!- r_linux [~r_linux@static.] has joined
19:04 < GeertJohan> thx :)
19:04 < GeertJohan> will look into GobEncoder too :)
19:05 < GeertJohan> brb/afk
19:06 < skelterjohn|work> don't know why list doesn't export its fields
19:06 -!- nekoh [~nekoh@p4FC70E54.dip0.t-ipconnect.de] has joined #go-nuts
19:17 -!- nekoh [~nekoh@p4FC70E54.dip0.t-ipconnect.de] has quit [Ping timeout: 255
19:19 -!- napsy [~luka@] has joined #go-nuts
19:22 * TheMue added a new article about dynamic method invocation to his 'Coding
in Go' series http://bit.ly/lWMkHT
19:24 < _nil> adg: you up?
19:27 < ww> GeertJohan: there are packages for red-black and avl trees and
19:27 < ww> (just look here so don't know the context but maybe useful to
19:27 -!- franciscosouza [~francisco@] has quit [Quit: franciscosouza]
19:30 -!- rlab [~Miranda@] has quit [Quit: Miranda IM! Smaller,
Faster, Easier.  http://miranda-im.org]
19:30 -!- ronnyy [~quassel@p4FF1C439.dip0.t-ipconnect.de] has quit [Remote host
closed the connection]
19:30 -!- franciscosouza [~francisco@] has joined #go-nuts
19:32 -!- foocraft [~ewanas@] has quit [Quit: Leaving]
19:33 -!- dreadlorde [~dreadlord@c-24-11-39-160.hsd1.mi.comcast.net] has joined
19:33 -!- chomp [~chomp@dap-209-166-184-50.pri.tnt-3.pgh.pa.stargate.net] has quit
[Quit: Leaving]
19:34 -!- tobym [~tobym@rrcs-208-105-11-82.nyc.biz.rr.com] has joined #go-nuts
19:34 -!- tvw [~tv@e176001230.adsl.alicedsl.de] has joined #go-nuts
19:40 -!- Ekspluati [3e4ed8ee@gateway/web/freenode/ip.] has quit
[Ping timeout: 252 seconds]
19:48 -!- cenuij [~cenuij@base/student/cenuij] has quit [Remote host closed the
19:50 -!- cenuij [~cenuij@] has joined #go-nuts
19:50 -!- cenuij [~cenuij@] has quit [Changing host]
19:50 -!- cenuij [~cenuij@base/student/cenuij] has joined #go-nuts
19:50 -!- frobnitz [~ian@king.bitgnome.net] has joined #go-nuts
19:51 -!- virtualsue [~chatzilla@nat/cisco/x-czktqnmzgmsogrul] has joined #go-nuts
19:52 < skelterjohn|work> potentially confusing source of compile error...
19:52 < skelterjohn|work> you have type IF interface { foo() }
19:53 < skelterjohn|work> and type A struct {...stuff...}; func (a *A) foo()
{ ...  }
19:53 < skelterjohn|work> so *A implements the interface IF
19:53 < skelterjohn|work> now if you do type B struct { A }
19:53 < skelterjohn|work> *B does not implement the interface IF
19:53 < KirkMcDonald> That makes sense.
19:54 < skelterjohn|work> yes
19:54 < skelterjohn|work> but it's not obvious
19:54 < skelterjohn|work> pre-diagnosis, anyway
19:54 -!- eimantas [~eimantas@ip-212-52-52-163.kava.lt] has quit [Quit: eimantas]
19:54 < skelterjohn|work> or at least, took me a solid couple of minutes :)
19:55 < skelterjohn|work> you can call foo() on something of type A, though
19:55 < skelterjohn|work> as long as that something is addressable
19:55 < skelterjohn|work> i kind of want that slippyness to work with the
embedding, too
19:56 < ww> doesn't it as long as you keep pointers and non-pointers
19:57 < skelterjohn|work> i'm not sure
19:57 < ww> i mean if you did func (a A) it should work, no?
19:57 < ww> B struct { *A }
19:58 < ww> maybe i'm wrong...
19:58 < skelterjohn|work> with func (a *A) and B struct { *A } that all
19:58 < skelterjohn|work> i guess...  with type B struct { A }, the A
component of B is not addressable?
19:58 < skelterjohn|work> calls for a test
19:59 -!- firwen [~firwen@2a01:e34:eea3:7e10:4a5b:39ff:fe51:e8ae] has quit [Remote
host closed the connection]
20:00 < skelterjohn|work> http://pastebin.com/Kbz4EDr0
20:00 < skelterjohn|work> that compiles
20:00 -!- tobym [~tobym@rrcs-208-105-11-82.nyc.biz.rr.com] has quit [Ping timeout:
260 seconds]
20:00 < skelterjohn|work> so maybe my original confusion comes from
something else
20:01 -!- zcram [~zcram@95-153-6-253.cdma.dyn.kou.ee] has joined #go-nuts
20:01 -!- tobym [~tobym@rrcs-208-105-11-82.nyc.biz.rr.com] has joined #go-nuts
20:02 -!- elephants [~elephants@] has joined #go-nuts
20:03 -!- rcrowley [~rcrowley@c-71-202-44-233.hsd1.ca.comcast.net] has quit [Quit:
Computer has gone to sleep.]
20:09 -!- niklas [~niklas@stgt-5f703506.pool.mediaWays.net] has joined #go-nuts
20:11 -!- keithcascio [~keithcasc@nat/google/x-paaptvyfntdpmbbj] has joined
20:12 < niklas> Hi all, can someone in here help me with a weired memory
size issue with gccgo?
20:13 <+iant> what is the issue?
20:13 -!- dlowe [~dlowe@ita4fw1.itasoftware.com] has joined #go-nuts
20:13 < niklas> I'm writing a system that works with graphs and I'm reading
in a graph describing all steets in Germany (based on OSM Data but in a compact
20:14 < niklas> It's about 23 000 000 vertices and 48 000 000 edges
20:15 < niklas> the graph is hold in a so called offset-array
representation, that is 3 Arrays, 1 for Vertices, 1 for Edges and 1 for Offsets,
the offset is simply an array of uints that hold for every vertex the postion of
it's first outgoing edge
20:15 < niklas> the vertex id simply being it's position in the vertex array
20:17 < niklas> note that this representation has been used a lot at our
university and it definitely works for graphs at around 1 million edges and 400
000 vertices
20:18 < niklas> the arrays in my case are made of structs Edges: Target
(uint), Weight (Float32 or uint both tested), Vertices: X pos (float32), Y pos
20:18 < skelterjohn|work> currently the length of a slice is limited to the
size of a 32 bit integer (just in case that is your issue)
20:19 < skelterjohn|work> since you're talking in the millions
20:19 < niklas> the weired thing now is the giant memory use (I've tested
with a smaller Graph (1 million edges 400 000 vertices)
20:19 < skelterjohn|work> oh - just using lots of memory
20:19 <+iant> how are you measuring memory use?
20:19 < niklas> it's all well below 4 billion
20:19 < niklas> htop
20:19 <+iant> which field are you looking at?
20:20 < skelterjohn|work> niklas: is it conceivable that your program would
ever want that much memory?  and you just expect the garbage collector to be doing
a better job?
20:20 < niklas> for the small graph virtual is at 150 MB and real at around
20:20 <+iant> is this x86?  32-bit or 64-bit?
20:20 < niklas> the graph is completely static at program runtime so there
is not much going on there
20:21 < niklas> this is round about the same for both + or minus some MB
20:22 < niklas> the problem is the same algorithm that works with a small
and medium graph gives me index out of bounds with the big graph and also simply
loading such a graph into RAM shouldn't take
20:22 < niklas> 1500 MB real
20:22 < niklas> even the big one I mean it's really just those 3 arrays I
20:22 -!- alehorst [~alehorst@] has quit [Quit: Leaving.]
20:23 < niklas> in the Algorithm (Dijkstra but also DFS) there is some map
usage and another array but even without having started the Alg at all I'm looking
at 1500 MB real
20:23 < niklas> and it's cleary growing with graph size
20:23 <+iant> you go from a million edges to 48 million edges are memory
increases 10-fold
20:23 <+iant> that doesn't seem so bad
20:24 <+iant> so you are also concerned about memory usage in the smaller
20:24 < niklas> also note that the big graph can be stored with pretty much
the same representation (though they can't use structs and ints only) with Java in
1500 mb ram
20:24 <+iant> how are you building the arrays?
20:24 <+iant> is this x86?  32-bit or 64-bit?
20:24 < niklas> make([]Vertex,numvertices)
20:24 < niklas> the big one is only 64bit
20:25 <+iant> you just said Java uses the same amount of memory?
20:25 < niklas> but shouldn't it be more like structsize*numVertices*2 +
20:25 < aiju> + memory management overhead
20:25 < crunge> is the amount of memory the program is using a problem?
20:26 <+iant> when I add up the amount of memory your data structures
require I get 568M
20:26 <+iant> so obviously I am missing something
20:26 < niklas> It somehow triggers the index out of bounds problem and yes
it's quite a bit more then I'd expect
20:26 <+iant> what is your calculation for required memory
20:27 < niklas> http://pastebin.com/QDMFtdb3 that's my structures
20:27 <+iant> what is the index out of bounds problem?
20:28 < niklas> 8 byte per Edge+ 8 byte per vertex + 4 byte per vertex for
offset (just read that uint is alwasy 32bit in Go)
20:28 < crunge> index out of bounds implies you're trying to access an
element beyond the end of an array/slice
20:29 -!- bmizerany [~bmizerany@] has joined #go-nuts
20:29 <+iant> so 48000000 * 8 + 23000000 * 8 + 23000000 * 4 ?
20:29 < skelterjohn|work> side note: if you want to use a 32 bit uint, use
20:29 <+iant> when I calculate that I get 660,000,000
20:29 < niklas> That doesn't work with Atoui
20:29 < skelterjohn|work> that shouldn't hold you back =p
20:30 < niklas> so where is the other gigabyte going?
20:30 <+iant> and you are measuring about twice that much memory usage?
20:30 < niklas> I'm measuring 1500 mb after freshly loading the graph
20:30 < niklas> no algorithm performed
20:30 < skelterjohn|work> if you measure twice that memory, bet you're doing
an extra allocation for each element you are storing
20:30 <+iant> seems that way to me too
20:31 <+iant> do you ever call append?
20:31 < skelterjohn|work> perhaps taking the address of something you don't
need to keep around?
20:31 <+iant> since memory is measured by htop, you are measuring total
allocation, not necessarily current allocation
20:31 < niklas> the weired thing is that for both smaller graphs I have my
DFS doesn't crash even if thrown at every possible vertex (have done it for the
middle sized one with 1 1/2 h long running no leaking or something)
20:31 < niklas> no append used before the algorithms start
20:32 < crunge> I'm a go newb, but if you're getting index out of bounds, I
think you have a logic error somewhere
20:32 <+iant> you should be able to run the program under the debugger to
see what the out of bounds index is
20:32 < skelterjohn|work> certainly the panic should tell you where it
20:32 <+iant> he's using gccgo, its unfortunately less sophisticated
20:32 < niklas> It just panics without any indication of where
20:33 < aiju> why does he use gccgo
20:33 <+iant> the debugger will tell you where
20:33 -!- binarii [~binary@] has quit [Remote host closed the
20:34 < niklas> the problem being as I said it only out of boundes with the
giant graph and only if the conected component becomes bigger, in the middle sized
graphs my DFS can explore more then 300 000 vertices in 1 second and as I said
running it on different vertexes for over an hour never produced an index out of
20:34 < skelterjohn|work> that doesn't make it impossible that something
about the larger dataset is wrong
20:34 <+iant> we can't possibly tell you why you are getting an index out of
bounds error; how could we?
20:35 <+iant> I am telling you to run it under the debugger to find out what
has gone wrong
20:35 < niklas> Yeah that's probably true
20:35 < skelterjohn|work> is wrong -> is subtly different and causes your
alg to fail
20:35 < niklas> can I debug the gccgo compiled one as well?
20:35 <+iant> sure
20:35 < niklas> I mean with gdb
20:35 < niklas> ok
20:35 <+iant> sure
20:35 <+iant> compile with the -g option to generate debug info
20:35 < niklas> how do I point gdb at the sources?
20:36 <+iant> if it can't just find them, set the "set directory" command
20:36 <+iant> it's just like in C/C++
20:36 < niklas> ok I see
20:38 < niklas> so let's see^^
20:39 -!- hcatlin [~hcatlin@pdpc/supporter/professional/hcatlin] has joined
20:39 < kevlar_work> iant, on an entirely unrelated note, would it take much
more than a few simple wrapper functions around Semacquire and Semrelease to make
a sync.Semaphore type?
20:39 < kevlar_work> and is that (too simple, doable in an external library
now) why it hasn't been done?
20:41 < aiju> maybe because Semacquire and Semrelease are already easy to
20:41 -!- rutkowski [~adrian@078088210082.walbrzych.vectranet.pl] has quit [Quit:
WeeChat 0.3.3-dev]
20:41 < aiju> and semaphores are evil spawn from hell?
20:42 -!- zozoR [~Morten@2906ds2-arno.0.fullrate.dk] has quit [Read error:
Operation timed out]
20:42 -!- virtualsue [~chatzilla@nat/cisco/x-czktqnmzgmsogrul] has quit [Quit:
ChatZilla 0.9.87 [Firefox 4.0.1/20110413222027]]
20:42 < niklas> by the way if I wanted to check what happens with 6g, I
encountered that it couldn't find a package for my graphrep that is in a subdir
and was compiled, and the -I option didn't help is there a special way to compile
a package?
20:43 < kevlar_work> niklas, -I has to specify a path to the directory
containing the .a file where the .a file has the same name as the package include
20:43 < kevlar_work> and in that case, the include cannot be relative (./)
20:43 < niklas> how do I generate a .a file?
20:44 < aiju> niklas: use the Make.pkg makefile?
20:44 < niklas> so I need to sue an absolute path?
20:44 < niklas> Can't I do it without a Makefile?  Have one gccgo specific
one as it seems the Go makefiles are only for gc
20:44 <+iant> kevlar_work: sure, it would be pretty easy to create
sync.Semaphore, it's just that nobody has done it
20:45 < kevlar_work> iant, I just figured that since nobody had, there was
some reason I shouldn't :)
20:45 < skelterjohn|work> aiju: does Make.pkg work for gccgo?
20:45 < kevlar_work> niklas, orly?  The only time I played with gccgo,
makefiles worked fine...
20:45 < aiju> no clue
20:45 < aiju> skelterjohn|work: he tries to use 6g
20:45 <+iant> skelterjohn|work: no, it doesn't
20:46 -!- dlowe [~dlowe@ita4fw1.itasoftware.com] has quit [Quit: Leaving.]
20:46 < kevlar_work> but then again, I think this was before the
make.pkg/make.cmd reorg, so.
20:46 < niklas> http://golang.org/doc/code.html#tmp_34 those Makefiles work
with gccgo?  How so?
20:46 -!- cco3 [~conleyo@nat/google/x-kcxkfuywdbgigths] has quit [Quit: Leaving.]
20:46 < niklas> I simply did some simple handcrafted ones
20:46 -!- cco3 [~conleyo@nat/google/x-buamjgocorcwkkvg] has joined #go-nuts
20:47 < kevlar_work> niklas, have you tried the new goinstall?
20:47 < skelterjohn|work> i find goinstall for local projects w/ $GOPATH to
be extremely clunky
20:47 -!- r_linux [~r_linux@static.] has quit
[Quit: Lost terminal]
20:47 < skelterjohn|work> also i doubt you can make goinstall use gccgo
20:48 < kevlar_work> can gb use gccgo?
20:48 < skelterjohn|work> no
20:48 < niklas> Ok back to my original problem, I tried running in gdb and
got this http://pastebin.com/4xdsUQV7
20:48 < kevlar_work> dang.
20:48 < skelterjohn|work> :)
20:48 < kevlar_work> what are the differences between gc and gccgo these
20:49 < niklas> gccgo got a different number of args for Open :-)
20:49 < niklas> os.Open that is
20:49 < skelterjohn|work> i'd happily accept a patch for gb that added gccgo
20:49 < skelterjohn|work> niklas, that is a library version issue, not a
compiler issue
20:49 <+iant> niklas: now type "where"
20:49 <+iant> niklas: have you ever used gdb before?
20:50 < niklas> Not for anything serious without a Gui
20:50 <+iant> well, "where" will give you a backtrace, "up" and "down" will
move you up and down the stack frame, "print x" will print the value of the
variable x in the current stack frame
20:50 < niklas> But I've dealt with stacktraces and so on just don't know
the commands
20:51 < kevlar_work> skelterjohn, I think all of the external builders (mine
included) will be obsoleted by the new standard builder
20:51 <+iant> "help" will give you help
20:51 < kevlar_work> skelterjohn, so if anything, it should get added to
20:51 -!- GeertJohan [~Squarc@ip4da06866.direct-adsl.nl] has quit [Quit: Leaving.]
20:51 < niklas> Ah I see I think I got it, even though it's reporting index
out of range it seems to be stack overflow
20:51 < kevlar_work> (if it's not there)
20:51 -!- bmizerany [~bmizerany@] has quit [Remote host closed the
20:51 < skelterjohn|work> kevlar_work: i've accepted that.  but so far it's
not impressive.
20:51 < skelterjohn|work> which one did you make?
20:52 <+iant> niklas: are you using the gold linker?
20:52 < kevlar_work> skelterjohn, gofr.
20:52 < niklas> http://pastebin.com/wUYtdGNL
20:52 <+iant> if you use gold, you should never get a stack overflow
20:52 < kevlar_work> It was, by my count, the first one to support cgo.
20:52 < kevlar_work> I stopped maintaining it when the community's ideas
about how to organize a project shifted and I got too busy with moving and
20:53 <+iant> niklas: is that more recursion than you expect for your
20:53 < kevlar_work> (it is still used as the basis for the compiler in
goclipse, though, iirc)
20:53 < niklas> Probably just a biger graph then expected^^ Should be easy
to port to an explicit stack
20:53 <+iant> niklas: again, the stack should never overflow
20:53 < niklas> just a pitty the error message was so misleading
20:53 < skelterjohn|work> kevlar_work: if i had been a bit more circumspect
I probably would have used it instead of writing gb
20:54 < skelterjohn|work> but i didn't look hard enough to see what already
20:54 < skelterjohn|work> gofr seems (from the readme) to do the same kind
of thing
20:54 < skelterjohn|work> as gb
20:54 < niklas> http://pastebin.com/RQjcUNVH it says --enable-gold but also
20:54 <+iant> niklas: it's quite unlikely that a stack overflow led to an
index out of bound error
20:55 < niklas> Hmm, but doesn't the trace from gdb look a lot like a stack
20:55 <+iant> niklas: no, it looks like a deeply recursive algorithm
20:55 <+iant> biab
20:56 < kevlar_work> skelterjohn, if I recall, you announced on the mailing
list very soon after I did.  I made mine because the only other offering didn't do
the kinds of relative includes that I wanted and didn't have gotest support, then
I added cgo for grins.
20:56 < skelterjohn|work> kevlar_work: your last commit is before my first
commit :)
20:56 < niklas> Good point *shy*
20:56 < kevlar_work> lol, maybe I'm thinking of a different one.
20:57 -!- iant1 [~iant@nat/google/x-twhbggfgbdotdrub] has joined #go-nuts
20:57 < skelterjohn|work> i began in january of this year
20:57 < kevlar_work> mine grew organically out of a makefile generator I had
20:58 < niklas> By the way did I mention it crashes with the same error for
two differen algorithms Dijkstra and DFS even though they both work flawlessly on
medium sized graphs
20:58 -!- sebastianskejoe [~sebastian@] has joined #go-nuts
20:59 -!- tobym [~tobym@rrcs-208-105-11-82.nyc.biz.rr.com] has quit [Ping timeout:
240 seconds]
20:59 < skelterjohn|work> maybe a list grew beyond the size of an int and
you index it with a type that can be larger?
20:59 < iant1> it is extremely unlikely that, even if a stack overflow does
occur, it would be reported as an out of bounds index error
20:59 < skelterjohn|work> an index error might make sense there
21:00 < kevlar_work> niklas, is it possible that you're indexing with a type
*smaller* than an int that is overflowing into the negatives?
21:00 -!- iant [~iant@nat/google/x-mtmodrzjenewzgms] has quit [Ping timeout: 260
21:00 < skelterjohn|work> oh - good one
21:00 < kevlar_work> skelterjohn|work, work with micros and hardware for
awhile and you begin to appreciate that 127+1=-128
21:01 < niklas> Nope give me a moment and I can pastebin the algorithm it's
like super simple 10 minute DFS
21:01 < ww> iant1: if you sit anywhere near the schema.org people, please
tell them that i'm cross because it looks like they're playing google vs.
facebook politics instead of building on extremely
similar-yet-not-quite-compatible open standards existing work
21:02 -!- ExtraSpice [XtraSpice@78-57-204-104.static.zebra.lt] has quit [Remote
host closed the connection]
21:02 < iant1> I know nothing about schema.org, sorry, I don't even know
whether Google is driving it--it's a cross-industry thing as far as I can see
21:02 -!- mode/#go-nuts [+v iant] by ChanServ
21:02 <+iant> does Facebook have something like schema.org?  I wonder why
21:03 < aiju> schema.org sounds xmlsy
21:03 < kevlar_work> ww, a lot of "open" initiatives aren't happy with
facebook because of how locked-down their data policies are.
21:03 < ww> no, but facebook decided (quite sensibly) to use rdfa, which is
a w3c thing
21:03 -!- robteix [~robteix@] has quit [Ping timeout: 260 seconds]
21:03 < aiju> "sensibly" and "w3c" don't belong in one sentence
21:03 <+iant> Google almost always choose open standards when possible, so I
suspect there was some reason that rdfa didn't work, or else this is being driven
by some other company and Google just signed onto it
21:04 <+iant> I don't know
21:04 -!- robteix [~robteix@] has joined #go-nuts
21:05 <+iant> ww: http://schema.org/docs/faq.html#14
21:05 < ww> iant: yes, i know
21:05 < ww> but see this response: manu.sporny.org/2011/false-choice/
21:06 < niklas> I really fail to see where it can crash with out of bound
especially since the only complicated part, that is filling the offset array was
before asking the user for the stating vertex and works
21:07 <+iant> ww: well, this is far beyond anything I know about
21:07 < niklas> and considering it works on medium sized graphs and produces
results verfied by several other independent solutions
21:08 < skelterjohn|work> niklas: did you find out what line the panic
occurred on?
21:08 < skelterjohn|work> alternatively, could you compile and run with 6g
instead?  (and have the panic report this information)
21:08 < niklas> it says waht is line 9 in the paste
21:08 <+iant> skelterjohn|work: he pasted it: http://pastebin.com/wUYtdGNL
21:08 -!- Count_Niedar [~bleh@ip68-99-166-222.hr.hr.cox.net] has quit [Quit: (
www.nnscript.com :: NoNameScript 4.22 :: www.esnation.com )]
21:08 < skelterjohn|work> my mistake
21:09 <+iant> although those lines don't map to the most recent paste
21:09 -!- Niedar [~bleh@ip68-99-166-222.hr.hr.cox.net] has joined #go-nuts
21:09 -!- werdan7 [~w7@freenode/staff/wikimedia.werdan7] has quit [Ping timeout:
620 seconds]
21:09 <+iant> I note that there is another large slice you didn't mention
earlier: visited
21:09 < skelterjohn|work> aha - that was my guess :)
21:09 < niklas> ah yeah just saw this as well
21:09 < skelterjohn|work> what line does the panic correspond to in the
pasted code?
21:10 < niklas> that I didn't mention it that is
21:10 < niklas> line 9
21:10 -!- angasule [~angasule@] has joined #go-nuts
21:10 < ww> iant: i guess the good news is the big search engines have
finally come round to the idea that we need structured data and that heuristics
and natural language techniques don't quite cut it...
21:10 < skelterjohn|work> sometimes panics can slip a line
21:10 < ww> ...  anyways back to on-topic golang chatter
21:10 < skelterjohn|work> might be from line 7
21:10 < skelterjohn|work> where you index into the visited slice
21:11 < niklas> "visited := make([]bool, len(graph.Vertices))"
21:11 <+iant> niklas: the out-of-bounds error is on the visited slice
21:11 < niklas> hmm I got an idea
21:12 <+iant> in gdb, go up to that stack frame and "print e"
21:12 <+iant> and "print visited"
21:12 < skelterjohn|work> most likely from line 5 actually
21:12 < niklas> maybe there is a target in the edges of the big graph that
is not actually a valid vertex
21:12 < niklas> Ah thanks guys I now know what's the problem
21:13 < skelterjohn|work> great!
21:13 < niklas> always be super humble when someone else sends you a new
file format that is quite like yours but not exactly
21:13 < niklas> Thank you a lot
21:13 < skelterjohn|work> keep using go
21:14 < niklas> I was so much thinking about the other graphs that worked
that I missed that this one might have something different apart from the size
21:14 < niklas> and that is, the other guy had vertices numbered 1 to n and
I have them 0 to n-1
21:14 < skelterjohn|work> i've been there
21:15 < skelterjohn|work> not specifically there
21:15 < skelterjohn|work> but that kind of place
21:15 <+iant> ouch
21:16 < niklas> And about the memory usage, you think it's normal that it
takes 1500 mb or do you think there is a problem hidden, I mean the pasted code is
quite straight forward concerning allocation
21:16 < skelterjohn|work> niklas: you might be surprised
21:17 < skelterjohn|work> taking the address of a variable causes it to be
garbage collected (extra allocation sometimes)
21:17 < skelterjohn|work> or storing it in an interface
21:20 < niklas> so might something like this line "readEdge(reader,
&sourceId, &graph.Edges[eCount])" for every edge cause the compiler to store extra
garbage collection information for it?  even though it's a struct in a slice that
lives on anyway?
21:21 <+iant> no, taking the address of an element of a slice does not have
any particular memory usage effects
21:21 <+iant> taking the address of sourceid will cause it to be
heap-allocated, but that is just one variable
21:22 < skelterjohn|work> do you allocate graph.Edges before you read
everything in?
21:22 < skelterjohn|work> or do you expand it as needed
21:22 < skelterjohn|work> if you paste code someone here could probably find
21:24 < niklas> it's at line 138 -> NewGraph() line 80
21:24 < niklas> a simple make
21:25 < ww> niklas: just fyi, there has been some recent work with
apparently quite good results on some types of graphs (edge-labelled graphs) using
techinques from column-store databases (like are found in some papers and thesis
and such around monetdb)
21:25 -!- dj2 [~dj2@] has quit [Remote host closed the connection]
21:25 -!- bytbox [~s@] has joined #go-nuts
21:26 < ww> depends what types of graph operations you want to do, but if
you can sort the (node, edge, node) in a certain way first, and use things like
rle, you can get it very compact and yet very fast
21:26 < ww> anectodally scales to some billions of edges
21:26 < ww> on relatively common hardware
21:27 < niklas> ww: sounds cool, actually as a Student of Software
Engineering at University of Stuttgart we are working for a project with some
leading Algorithmics Researches speciallized in Shortest-Path problems as well as
Discrete Optimisation some amazing work is going on there as well, especially with
Contraction hiearchies and so on
21:28 < skelterjohn|work> niklas: how does memory look if you stop after
calling NewGraph() that is, don't read any data in
21:28 < ww> Algorithmics the risk management people from toronto?
21:28 < skelterjohn|work> just those several make() calls
21:28 -!- dreadlorde [~dreadlord@c-24-11-39-160.hsd1.mi.comcast.net] has quit
[Ping timeout: 260 seconds]
21:28 -!- Fish- [~Fish@9fans.fr] has quit [Quit: So Long, and Thanks for All the
21:28 < crunge> ww: Algorithmics is the band Anne Lennox was in
21:29 < niklas> No Algorithms Department of the University of Stuttgart
21:29 < niklas> Hehe right bad translation from German
21:30 < ww> :)
21:30 < skelterjohn|work> algorithmics might be best translated as algorithm
21:30 -!- thomas_b [~thomasb@cm-] has quit [Quit:
21:30 < skelterjohn|work> that's what they call it in my dept anyway
21:30 < ww> you should go up and visit with the CWI people in the
netherlands and hear about column stores
21:31 < niklas> hah an easy albeit bad hack (decrementing all values read in
from the file of my colleague) solved it and the DFS now takes 9 seconds and
explroes 23040139 vertices
21:32 < niklas> so good work guys on the how far can we do recursion front
21:32 -!- marko_ [~marko@host213-200-dynamic.36-79-r.retail.telecomitalia.it] has
quit [Ping timeout: 250 seconds]
21:32 < skelterjohn|work> keep any stats on the max depth?
21:32 < ww> this is actually a big problem in my field - a lot of EU and
national government research money funding people to create enormous amounts of
graph-shaped data...  and no really good way to store it in a queriable way long
21:33 < ww> this bears on the schema.org stuff - because what you get is
graph-shaped data...
21:33 < ww> though the search engine companies can just throw hardware at
the problem
21:33 < niklas> note that this program is actually more like a prototype to
get me started in Go and prepared for our Semester project which is a
Server/Client infrastructure doing approximation of TSPs on OpenStreetMap data
that can for example be used to plan a route for a salesmen *doh* and several
other approximations of NP hard Graph problems
21:33 -!- marko_ [~marko@host213-200-dynamic.36-79-r.retail.telecomitalia.it] has
joined #go-nuts
21:33 < ww> which is not necessarily sustainable for long-term library-like
hosting of research output
21:35 -!- delinka [~delinka@pidsley.praxxium.com] has joined #go-nuts
21:35 -!- jcao219 [~Jimmy@] has joined #go-nuts
21:35 -!- jcao219 [~Jimmy@] has left #go-nuts ["Leaving"]
21:35 < niklas> haven't measured the max depth now but sure will in the
future, it's actually part of a programming task for the students I'm tutoring
(I'm in my fourth semester) and things like measuring the longest shortest path in
a graph
21:35 < ww> http://linkedgeodata.org/ is one such project (Soeren Auer's
group in Leipzig) making graph-shaped data out of OSM
21:36 < ww> (unfortunately their site seems down at the moment, go figure)
21:36 < skelterjohn|work> ww: probably a router issue
21:36 < skelterjohn|work> couldn't find the best path to the host, something
21:36 < skelterjohn|work> nevermind
21:37 < niklas> ;-)
21:38 -!- pjacobs [~pjacobs@] has joined #go-nuts
21:38 < niklas> Semantic Web is an awesome topic just a month or so ago I
was at a talk by Denny Vrandecic of KIT (Karlsruhe) who is one of the guys behind
the semantic web version of MediaWiki
21:39 -!- marko_ [~marko@host213-200-dynamic.36-79-r.retail.telecomitalia.it] has
quit [Ping timeout: 255 seconds]
21:39 < niklas> and things like DBPedia
21:40 < ww> yes, which is why schema.org is such a shame - they could have
very easily just joined up with all that and instead decided to go their own way
21:40 * ww will stop complaining about that here now
21:40 < niklas> hehe
21:42 -!- Venom_X [~pjacobs@75-27-133-72.lightspeed.austtx.sbcglobal.net] has quit
[Ping timeout: 240 seconds]
21:42 < niklas> By the way did anyone of you guys ever notice that pretty
much every site (like Wikipedia) claims you can do Change-Key in a standard heap
in O(log n) but no one actually implements it not even Javas standard priority can
change the key while queued
21:42 < niklas> the problem being that one never knows the postion of an
element in the heap because it's shifted by other insertions
21:43 < bytbox> niklas: yeah.  You never actually need it.  (Until you need
21:44 < niklas> had to augment the heap with a map to get from vertex id to
the postion in the heap, that was O(log n) but because I had to keep track of
every swap and change-key hapens not too often in street graphs I decided to just
21:44 < niklas> probably will go for fibonacci heaps in the real software^^
21:48 -!- TheMue [~TheMue@p5DDF52EC.dip.t-dialin.net] has quit [Quit: TheMue]
21:49 < ww> i wonder...  idle thought...  if you were to assume very large
ram (something like n^2 + cn in the number of nodes where c is related to the
average path length) how well an algorithm like the one used in BGP would do for
getting you all shortest paths...
21:49 < ww> (large ram or striped ssd...)
21:51 < niklas> hmm dunno, back to a lot of RAM interesting observation when
returning just after allocation the graph with NewGraph I get a virtual memory use
of 689 MB pretty much what we calculated
21:54 <+iant> how deep does that recursion get?
21:54 <+iant> new stack frames are going to count against your virtual
memory size
21:54 < niklas> after only reading the nodes (which unexpectedly fast
(reading uints was CPU bound for the edges) I'm at 1500 MB virt and 900 real
21:54 < niklas> no recursion yet
21:54 <+iant> oh, so it's reading
21:55 < niklas> the 1500 MB mentioned in the beginning were after
construction the graph before running any recursive algorithm
21:55 <+iant> k
21:55 -!- gtaylor [~gtaylor@99-126-136-139.lightspeed.gnvlsc.sbcglobal.net] has
quit [Quit: gtaylor]
21:56 < ww> has there been anything done in go about tail recursion?
21:56 < niklas> it's really inetresting I'm exeting what is line 146 in the
paste http://pastebin.com/jEsV0qaG and it has a REAL usage of 1000 MB twice as
much as allocated
21:56 -!- bmizerany [~bmizerany@] has joined #go-nuts
21:56 -!- bmizerany [~bmizerany@] has quit [Remote host closed the
21:56 -!- bmizerany [~bmizerany@] has joined #go-nuts
21:57 < niklas> and at line 139 it was still like 680 MB Virtual
21:57 < niklas> so my guess something is weired with the reading data in
21:57 -!- niemeyer [~niemeyer@201-40-174-53.pltce701.dsl.brasiltelecom.net.br] has
quit [Ping timeout: 240 seconds]
21:58 < niklas> I'd guess the strings from strings.Fields didn't get
21:59 -!- niemeyer [~niemeyer@201-40-174-53.pltce701.dsl.brasiltelecom.net.br] has
joined #go-nuts
21:59 < niklas> the comment at line 142 is what I used before but FScanln is
just really slow
21:59 -!- robteix [~robteix@] has quit [Ping timeout: 255 seconds]
22:00 -!- jstemmer [~cheetah@mrpwn.stemmertech.com] has quit [Quit: leaving]
22:01 <+iant> strings.Fields doesn't do any allocation, but it is possible
that the string returned by r.ReadString is not getting collected for some reason
22:03 < niklas> by the way I keep it running in the background and even
though there is absolutely nothing happening it sits there at 1023 MB real ram use
22:04 -!- franciscosouza [~francisco@] has quit [Quit: franciscosouza]
22:05 -!- replore_ [~replore@ntkngw256114.kngw.nt.ftth.ppp.infoweb.ne.jp] has
joined #go-nuts
22:06 -!- iant1 [~iant@nat/google/x-scsurwkriibnrnzu] has joined #go-nuts
22:06 -!- iant [~iant@nat/google/x-twhbggfgbdotdrub] has quit [Read error:
Connection reset by peer]
22:07 -!- mode/#go-nuts [+v iant] by ChanServ
22:08 -!- replore_ [~replore@ntkngw256114.kngw.nt.ftth.ppp.infoweb.ne.jp] has quit
[Remote host closed the connection]
22:09 < ww> fwiw, i'd be inclined to implement the graph like this:
22:09 <@adg> _nil: yes
22:09 < ww> but the fact that uint is 32 bits (right?) would probably make
the pointer version quite a bit bigger
22:10 < ww> otoh, it's easily mutable
22:13 -!- mogoh [~mogoh@ip-95-222-106-192.unitymediagroup.de] has joined #go-nuts
22:14 < mogoh> hi
22:14 < mogoh> how do i get the length of a string?
22:14 < niklas> Yeah, this presentation is primarily designed for Java
(which we might use later on) where one can't store structured information without
much overhead, I think since my interface is quite general I will just try out
differen implementations how they compare.  And thats one of the giant advantages
of Go I can in fact design a nice interface without much overhead while in Java I
even have to care that for all 2 dimensional arrays (a x b) a<<
22:14 < niklas> len(somestring)
22:15 < mogoh> niklas: i meant the number of charakters, not bytes
22:15 < niklas> @ww: The map is not needed if the vertex ids are simply the
indizes in the array
22:15 < mogoh> len("ß") == 2
22:15 -!- pjacobs [~pjacobs@75-27-133-72.lightspeed.austtx.sbcglobal.net] has
joined #go-nuts
22:15 < str1ngs> no 1
22:17 < ww> len(strings.Split(s, "", -1)) // not a very good answer
22:17 -!- Venom_X [~pjacobs@] has quit [Ping timeout: 250 seconds]
22:17 < str1ngs> len("?") == 1
22:18 -!- elephants [~elephants@] has quit [Remote host closed the
22:19 < mogoh> str1ngs: maybe an irc-encoding-problem
22:19 < ww> niklas: i guess it depends on if you want to add or remove nodes
after building the initial graph, or merge two graphs
22:19 < mogoh> str1ngs: i wrote an non-ascii char
22:19 < ww> mogoh: strings.Split will split each utf-8 rune
22:19 < ww> but i don't know if there's a better way
22:21 -!- wjlroe_ [u1877@gateway/web/irccloud.com/x-zropbpzyvxlcuvev] has joined
22:21 -!- mgray [~mgray@li226-224.members.linode.com] has joined #go-nuts
22:22 -!- wjlroe [~will@] has quit [Quit: leaving]
22:22 < niklas> ww: yes and today one of the researches mentioned that the
offset-array implementation is worse with cache use when multithreaded
22:23 < fvbommel> mogoh: utf8.RuneCountInString("ß") should work.
22:23 < niklas> ww: Do you have any guess why the reading of Nodes doubles
my RAM usage and it doesn't get collected
22:24 < niklas> the code looks quite harmless but it seems not to be
22:24 -!- zcram [~zcram@95-153-6-253.cdma.dyn.kou.ee] has quit [Quit: Leaving]
22:24 < str1ngs> mogoh: strange I'm using urxvt.  sorry about that.
22:25 < mogoh> fvbommel: thanks, that was, what i've searched for :)
22:25 < mogoh> ww, str1ngs : thanks also :)
22:27 < niklas> maybe because I use gccgo
22:27 < niklas> and bufio
22:28 < ww> does it not get collected or just not get returned to the os?
22:28 < ww> if you try allocating stuff afterwards, does it reuse some of
what is already allocated or does it go and allocate yet more?
22:29 < niklas> I don't know what exactlyu happens, I just have the program
sitting there waiting for user input reportedly (htop) using 1020 MB of real ram
22:29 < ww> i'm not sure the gccgo behaviour, but 6g will tend to hang onto
ram from the os perspective, but will re-use it
22:30 < ww> after you've read everything in, try allocating a big slice or
something and see if the ram goes up more
22:30 < niklas> maybe I could use non buffered io and get away with much
less RAM.  Looking at the bufi.ReadBytes/String code it looks like quite a task
for garbage collection
22:32 -!- photron [~photron@port-92-201-145-30.dynamic.qsc.de] has quit [Quit:
22:35 < niklas> well anyway thanks for all the help will look into this in
the coming days when I have time have a nice day
22:36 -!- niklas [~niklas@stgt-5f703506.pool.mediaWays.net] has quit [Quit:
22:39 -!- aat [~aat@rrcs-184-75-54-130.nyc.biz.rr.com] has quit [Ping timeout: 252
22:42 -!- hcatlin [~hcatlin@pdpc/supporter/professional/hcatlin] has quit [Read
error: Connection reset by peer]
22:43 -!- napsy [~luka@] has quit [Ping timeout: 240 seconds]
22:46 < mogoh> good night
22:46 -!- mogoh [~mogoh@ip-95-222-106-192.unitymediagroup.de] has quit [Quit:
22:47 -!- Bigbear1 [~Cody@d173-181-43-12.abhsia.telus.net] has joined #go-nuts
22:48 -!- tvw [~tv@e176001230.adsl.alicedsl.de] has quit [Remote host closed the
22:55 -!- bytbox [~s@] has quit [Remote host closed the connection]
22:56 -!- rcrowley [~rcrowley@c-71-202-44-233.hsd1.ca.comcast.net] has joined
23:00 -!- Venom_X [~pjacobs@75-27-133-72.lightspeed.austtx.sbcglobal.net] has quit
[Ping timeout: 244 seconds]
23:01 -!- dfc [~dfc@124-149-49-45.dyn.iinet.net.au] has quit [Quit: dfc]
23:03 -!- chressie [~chressie@dreggn.in-ulm.de] has quit [Ping timeout: 240
23:10 -!- Project-2501 [~Marvin@] has quit [Quit: E se abbasso questa
leva che succ...]
23:15 -!- TheCritic [~TheCritic@c-24-30-34-40.hsd1.ga.comcast.net] has joined
23:17 -!- dj2 [~dj2@CPE001f5b35feb4-CM0014048e0344.cpe.net.cable.rogers.com] has
joined #go-nuts
23:21 -!- iant [~iant@nat/google/x-scsurwkriibnrnzu] has quit [Ping timeout: 248
23:26 -!- tobym [~tobym@cpe-72-229-2-6.nyc.res.rr.com] has joined #go-nuts
23:31 -!- pharris [~Adium@rhgw.opentext.com] has quit [Quit: Leaving.]
23:31 -!- tobym [~tobym@cpe-72-229-2-6.nyc.res.rr.com] has quit [Ping timeout: 250
23:31 -!- TheCritic_ [~TheCritic@c-24-30-34-40.hsd1.ga.comcast.net] has joined
23:31 -!- iant [~iant@] has joined #go-nuts
23:32 -!- mode/#go-nuts [+v iant] by ChanServ
23:33 -!- TheCritic [~TheCritic@c-24-30-34-40.hsd1.ga.comcast.net] has quit [Ping
timeout: 248 seconds]
23:38 -!- nsf [~nsf@jiss.convex.ru] has joined #go-nuts
23:46 < TheCritic> wow, just discovered io/ioutil ....  that will make life
23:50 < nsf> :D
23:50 -!- robteix [~robteix@201-213-183-45.net.prima.net.ar] has joined #go-nuts
23:51 < TheCritic> I want to google search for some go code that implements
hash/crc64 ...  any suggestions on terms I should be using?  golang crc64 isnt
turning up much
23:52 < nsf> what do you mean "implements"?  uses maybe?
23:52 < nsf> and why do you need an example of using a hash function?  :\
23:53 < TheCritic> ah, I want to pass it a fully qualified path and get back
the 64 bit crc
23:53 < TheCritic> the way I have been learning is by reading the packages
code for examples
23:53 -!- dfc [~dfc@eth59-167-133-99.static.internode.on.net] has joined #go-nuts
23:54 < nsf> well, you can read crc64_test.go
23:54 < TheCritic> so, it would be great if I could search the go packages
to find one that imports crc64
23:54 < nsf> http://golang.org/src/pkg/hash/crc64/crc64_test.go
23:54 < TheCritic> ah, ok, reading
23:54 < nsf> it doesn't import it, but certainly using it
23:54 < nsf> uses*
23:59 < TheCritic> That really helps
--- Log closed Wed Jun 08 00:00:02 2011