hadoop - Does combiner runs conditionally -

-

min.num.spills.for.combine (default 3)

what signify?

a) min no. of map spills have combiner run? though have specified combiner, not guaranteed run?

b) min no. of spills have before combiner runs on merged/sorted single file created via io.sort.factor. each time new file created merging, combiner runs onto it, provided no. of spills min 3

i feel correct answer a) , can confirm that.

when map function generate intermediate result , first sent them buffer, partitioning , sorting start , , if combiner specified, invoked @ time. process in parallel map function. when map function finishes, spills on disk merged, , combiners invoked @ time too.

the buffer threshold limited io.sort.spill.percent, during spills created. if number of spills more min.num.spills.for.combine, combiner gets invoked on spills created before writing disk.

so answer question: right choice a) .

ref : this mail thread.


Comments

Post a Comment

Popular posts from this blog

c++ - Creating new partition disk winapi -

-
i'am trying create new partition , mount volume new, thought createfile let me this, code : lpctstr lpfilename=l"\\\\.\\device\\harddisk0\\partition3"; handle handl=createfile( lpfilename, generic_read | generic_write, file_share_read | file_share_write, null, create_always, file_attribute_normal, null ); if (handl==invalid_handle_value) { qdebug()<<"handl invalid"<<" error"<<getlasterror();} bool success = definedosdevice(ddd_raw_target_path,l"i:",l"\\device\\harddisk0\\partition3"); if(!success) qdebug()<<" definedosdevice failed "<<getlasterror(); bflag = getvolumenameforvolumemountpoint( l"i:\\", // input volume mount point or directory /** u in dir
Read more

Android Prevent Bluetooth Pairing Dialog -

-
i'm developing internal application uses bluetooth printing. want bluetooth pairing occur without user input. have managed working trapping android.bluetooth.device.action.pairing_request broadcast. in broadcast receiver call setpin method, , pairing works ok, bluetoothpairingdialog displayed second or two, disappears - see link below. https://github.com/android/platform_packages_apps_settings/blob/master/src/com/android/settings/bluetooth/bluetoothpairingdialog.java since broadcast non-ordered, can't call abortbroadcast() , , wondering if there other way prevent pairing dialog appearing. can hook window manager in way? i haven't been able come way without modifying sdk. if you're oem, it's easy (i'm on 4.3): in packages/apps/settings/androidmanifest.xml, comment intent filter pairing dialog: <activity android:name=".bluetooth.bluetoothpairingdialog" android:label="@string/bluetooth_pairing_request"
Read more

php - joomla get content in onBeforeCompileHead function -

-
i'm developing joomla system plugin. need add javascript , css files if [myshortcode] shortcode found in content. how can content in onbeforecompilehead function function onbeforecompilehead() { $document = jfactory::getdocument(); //add scripts if shortcode found ... $document->addstylesheet($cssfile, 'text/css', null, array()); the $content = jresponse::getbody(); not work. thanks try this, for style sheet, $document = jfactory::getdocument(); $document->addstylesheet('http://domain.com/templates/css/style.css'); for javascript file $document = jfactory::getdocument(); $document->addscript('full url'); also can this, $document = jfactory::getdocument(); // add javascript $document->addscriptdeclaration(' window.event("domready", function() { alert("an inline javascript declaration"); }); '); // add styles $style = 'body {'
Read more