performance - Why is Linq that slow (see provided examples) -

this linq slow:

ienumerable<string> iedrdatarecordids = dt1.asenumerable()     .where(x => x.field<string>(inputdataset.column_arguments_name) == sargumentname           && x.field<string>(inputdataset.column_arguments_value) == sargumentvalue)     .select(x => x.field<string>(inputdataset.column_arguments_recordid));  ienumerable<string> iedrdatarecordids_filtered = dt2.asenumerable()     .where(x => iedrdatarecordids.contains(                  x.field<string>(inputdataset.column_datarecordfields_recordid))               && x.field<string>(inputdataset.column_datarecordfields_field)                   == sdatarecordfieldfield               && x.field<string>(inputdataset.column_datarecordfields_value)                   == sdatarecordfieldvalue)     .select(x => x.field<string>(inputdataset.column_datarecordfields_recordid));  ienumerable<string> ievalue = dt2.asenumerable()     .where(x => x.field<string>(inputdataset.column_datarecordfields_recordid)                  == iedrdatarecordids_filtered.firstordefault()              && x.field<string>(inputdataset.column_datarecordfields_field) == sfieldname)     .select(x => x.field<string>(inputdataset.column_datarecordfields_value));  if (!ievalue.any()) //very slow @ point     return iedrdatarecordids_filtered.firstordefault(); 

this change accelerates factor of 10 or more

string srecordid = dt2.asenumerable()     .where(x => iedrdatarecordids.contains(             x.field<string>(inputdataset.column_datarecordfields_recordid))          && x.field<string>(inputdataset.column_datarecordfields_field)              == sdatarecordfieldfield          && x.field<string>(inputdataset.column_datarecordfields_value)              == sdatarecordfieldvalue)     .select(x => x.field<string>(inputdataset.column_datarecordfields_recordid))     .firstordefault();  ienumerable<string> ievalue = dt2.asenumerable()    .where(x => x.field<string>(inputdataset.column_datarecordfields_recordid) == srecordid          && x.field<string>(inputdataset.column_datarecordfields_field) == sfieldname)     .select(x => x.field<string>(inputdataset.column_datarecordfields_value));  if (!ievalue.any()) //very fast @ point     return iedrdatarecordids_filtered.firstordefault();  

the change store result directly in new variable , use create clause value instead of linq query (which should calculated when needed). linq seems calculate in bad way here or doing wrong?

here values of data

dt1.rows.count                     142  dt1.columns.count                    3  dt2.rows.count                     159  dt2.columns.count                    3  iedrdatarecordids.count()            1  iedrdatarecordids_filtered.count()   1  ievalue.count()                      1 

you're asking why

ienumerable<string> iedrdatarecordids_filtered = data;     foreach (var item in collection) {     //     iedrdatarecordids_filtered.firstordefault(); } 

is slower

string srecordid = data.firstordefault(); foreach (var item in collection) {     //     srecordid; } 

very because you're evaluating iedrdatarecordids collection every time firstordefault. isn't concrete object, it's enumerable set. that's function returns objects. every time query function called , you'll pay execution cost.

if change

ienumerable<string> iedrdatarecordids_filtered = dt2.asenumerable()...  var recordids = iedrdatarecordids_filtered.tolist(); 

and use recordids.firstordefault() you'll see huge performance increase.