mapreduce - XML data via API to Land in Hadoop -

-

we receiving huge amounts of xml data via api. in-order handle large data set, planning in hadoop.

needed in understanding how efficiently bring data hadoop. tools available ? there possibility of bringing data real-time ?

please provide inputs.

thanks help.

since receiving huge amounts of data, appropriate way, imho, use aggregation tool flume. flume distributed, reliable, , available service efficiently collecting, aggregating, , moving large amounts of data hadoop cluster different types of sources.

you can write custom sources based on needs collect data. might fins link helpful started. presents custom flume source designed connect twitter streaming api , ingest tweets in raw json format hdfs. try similar xml data.

you might wanna have @ apache chukwa same thing.

hth


Comments

Post a Comment

Popular posts from this blog

c++ - Creating new partition disk winapi -

-
i'am trying create new partition , mount volume new, thought createfile let me this, code : lpctstr lpfilename=l"\\\\.\\device\\harddisk0\\partition3"; handle handl=createfile( lpfilename, generic_read | generic_write, file_share_read | file_share_write, null, create_always, file_attribute_normal, null ); if (handl==invalid_handle_value) { qdebug()<<"handl invalid"<<" error"<<getlasterror();} bool success = definedosdevice(ddd_raw_target_path,l"i:",l"\\device\\harddisk0\\partition3"); if(!success) qdebug()<<" definedosdevice failed "<<getlasterror(); bflag = getvolumenameforvolumemountpoint( l"i:\\", // input volume mount point or directory /** u in dir
Read more

Android Prevent Bluetooth Pairing Dialog -

-
i'm developing internal application uses bluetooth printing. want bluetooth pairing occur without user input. have managed working trapping android.bluetooth.device.action.pairing_request broadcast. in broadcast receiver call setpin method, , pairing works ok, bluetoothpairingdialog displayed second or two, disappears - see link below. https://github.com/android/platform_packages_apps_settings/blob/master/src/com/android/settings/bluetooth/bluetoothpairingdialog.java since broadcast non-ordered, can't call abortbroadcast() , , wondering if there other way prevent pairing dialog appearing. can hook window manager in way? i haven't been able come way without modifying sdk. if you're oem, it's easy (i'm on 4.3): in packages/apps/settings/androidmanifest.xml, comment intent filter pairing dialog: <activity android:name=".bluetooth.bluetoothpairingdialog" android:label="@string/bluetooth_pairing_request"
Read more

VBA function to include CDATA -

-
i have following function include cdata: function cdatasection() dim objdom domdocument dim objkmlrootelement ikmldomelement dim objkmlelement ikmldomelement dim cdata ikmldomcdatasection set objdom = new domdocument set objkmlrootelement = objdom.createelement("balloonstyle") objdom.appendchild objkmlrootelement set objkmlelement = objdom.createelement("text") objkmlrootelement.appendchild objkmlelement set cdata = objdom.createcdatasection("text") cdata.data = "<![cdata[<b>latitude = $[latitude]</b>?]]>;" end function when run above, getting error "user defined data type not found" function. yes ripster says find library reference. vba window tools>references for list of relevant references see... http://msdn.microsoft.com/en-us/library/windows/desktop/ms763701%28v=vs.85%29.aspx however, if have hard time tracking down reference, replace dim sta
Read more