You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-06-06 - 17:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel

x86 Intel Atom D510 @1667 MHz, Linux 3.12.61-rt81 (Profile)

Latency plot of shadow in rack #1, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Tue Jun 06, 2023 12:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
23659910633,13cyclictest0-21swapper/210:12:482
2345991010,5rtkit-daemon2344-21rtkit-daemon05:07:281
2345991010,5rtkit-daemon2344-21rtkit-daemon05:07:281
2357999320,12cyclictest0-21swapper/009:40:210
2357999228,55cyclictest0-21swapper/009:42:440
236399915,78cyclictest0-21swapper/105:33:561
2357999136,47cyclictest19406-21sed07:57:400
2357999136,47cyclictest19406-21sed07:57:400
2357999122,12cyclictest0-21swapper/007:07:160
235799905,28cyclictest0-21swapper/005:07:390
235799905,28cyclictest0-21swapper/005:07:390
235799905,26cyclictest0-21swapper/005:37:040
235799904,26cyclictest0-21swapper/008:22:140
235799903,26cyclictest0-21swapper/009:03:110
235799885,76cyclictest0-21swapper/009:47:330
235799885,76cyclictest0-21swapper/009:47:330
235799884,26cyclictest0-21swapper/009:57:100
2357998827,54cyclictest0-21swapper/009:07:400
2357998827,54cyclictest0-21swapper/009:07:400
2357998826,55cyclictest0-21swapper/006:28:040
2357998817,13cyclictest0-21swapper/006:12:580
236399874,52cyclictest0-21swapper/110:00:071
235799875,73cyclictest0-21swapper/005:37:550
235799874,76cyclictest0-21swapper/009:27:430
235799874,76cyclictest0-21swapper/008:07:230
2357998722,9cyclictest17254-21unixbench_multi10:07:480
2357998722,13cyclictest0-21swapper/008:17:130
2357998717,13cyclictest0-21swapper/009:12:510
235799865,75cyclictest0-21swapper/006:22:580
235799864,27cyclictest0-21swapper/006:57:100
235799864,24cyclictest0-21swapper/007:32:220
2357998628,50cyclictest10133-21chrt05:22:440
2357998617,13cyclictest0-21swapper/008:37:250
235799855,74cyclictest0-21swapper/008:32:460
235799855,74cyclictest0-21swapper/006:57:430
235799854,25cyclictest0-21swapper/007:45:500
236399844,9cyclictest23018-21runrttasks08:06:501
235799845,27cyclictest0-21swapper/008:27:540
235799845,26cyclictest0-21swapper/010:27:350
235799845,26cyclictest0-21swapper/006:12:080
235799844,27cyclictest0-21swapper/010:17:440
2357998423,12cyclictest879-21crond10:27:080
2357998419,9cyclictest15335-21latency_hist10:07:090
2357998418,14cyclictest0-21swapper/005:17:530
236399834,72cyclictest0-21swapper/108:57:471
236399834,72cyclictest0-21swapper/108:42:331
236399833,11cyclictest0-21swapper/106:47:491
235799834,28cyclictest0-21swapper/006:42:320
2357998322,13cyclictest0-21swapper/008:42:440
2357998318,9cyclictest102350irq/46-eth007:47:520
236399823,8cyclictest27892-21timerwakeupswit05:57:571
236399823,72cyclictest0-21swapper/107:42:151
236399823,72cyclictest0-21swapper/105:52:201
236399823,72cyclictest0-21swapper/105:52:201
235799825,70cyclictest0-21swapper/007:26:520
235799823,10cyclictest29567-21idleruntime-cro10:37:080
235799823,10cyclictest29567-21idleruntime-cro10:37:080
2357998226,47cyclictest0-21swapper/007:12:180
2357998226,47cyclictest0-21swapper/007:12:180
2357998221,10cyclictest1199-21runrttasks05:53:170
2357998220,12cyclictest0-21swapper/009:32:330
2357998218,9cyclictest1199-21runrttasks05:14:560
236399814,70cyclictest0-21swapper/107:07:581
236399813,9cyclictest27050-21runrttasks10:30:321
236399813,8cyclictest17703-21grep09:02:371
235799815,18cyclictest0-21swapper/010:12:370
2357998122,10cyclictest1199-21runrttasks09:57:230
2357998120,13cyclictest0-21swapper/007:17:510
2357998113,13cyclictest0-21swapper/007:52:580
2357998111,13cyclictest0-21swapper/007:37:430
236399803,71cyclictest0-21swapper/108:38:011
236399803,69cyclictest0-21swapper/106:59:521
235799804,13cyclictest17035-21kworker/u8:006:17:500
2357998021,9cyclictest1199-21runrttasks08:54:240
2357998019,13cyclictest0-21swapper/009:17:340
2357998017,12cyclictest16825-21crond09:02:110
236399794,67cyclictest0-21swapper/107:27:391
236399794,67cyclictest0-21swapper/106:35:521
236399793,9cyclictest586-21runrttasks06:11:001
236399793,9cyclictest22755-21if_err_eth009:12:421
235799796,15cyclictest0-21swapper/005:29:560
235799796,12cyclictest0-21swapper/005:42:370
235799794,15cyclictest0-21swapper/005:47:410
235799794,15cyclictest0-21swapper/005:47:410
235799793,27cyclictest26544-21munin-node09:22:380
235799793,24cyclictest28711-21timerandwakeup07:07:580
2357997925,9cyclictest29110-21fschecks_count06:02:470
2357997922,11cyclictest18200-21grep06:47:410
2357997921,10cyclictest1199-21runrttasks06:40:040
2357997915,12cyclictest0-21swapper/008:22:300
236399784,50cyclictest0-21swapper/109:32:401
236399783,8cyclictest1199-21runrttasks10:13:591
236399783,68cyclictest0-21swapper/106:28:011
236399783,10cyclictest25341-21taskset08:11:151
235799783,12cyclictest0-21swapper/008:50:470
2357997817,13cyclictest0-21swapper/008:02:400
236399774,8cyclictest11835-21munin-run05:27:191
236399773,8cyclictest26299-21crond09:22:111
236399773,8cyclictest15736-21entropy06:42:361
236399773,8cyclictest10605-21idleruntime-cro09:57:091
236399773,67cyclictest0-21swapper/107:34:321
2363997716,45cyclictest29538-21munin-node07:12:421
2363997716,45cyclictest29538-21munin-node07:12:421
235799775,12cyclictest0-21swapper/007:27:360
235799773,21cyclictest0-21swapper/005:58:160
236399767,60cyclictest0-21swapper/107:22:451
236399763,8cyclictest26924-21pluginstate07:07:161
236399763,66cyclictest0-21swapper/110:02:481
236399763,66cyclictest0-21swapper/108:12:451
236399763,57cyclictest0-21swapper/109:37:321
2357997620,9cyclictest10848-21df_abs06:32:360
236399756,45cyclictest234682grep10:22:321
236399754,64cyclictest0-21swapper/110:20:201
236399753,9cyclictest5608-21munin-run06:22:181
236399753,9cyclictest4650-21taskset08:32:561
236399753,9cyclictest2534-21runrttasks08:31:081
236399753,8cyclictest13324-21iostat_ios08:52:381
236399753,66cyclictest0-21swapper/109:29:231
236399744,62cyclictest0-21swapper/109:47:451
236399744,62cyclictest0-21swapper/109:47:451
236399744,62cyclictest0-21swapper/105:42:551
236399743,9cyclictest6845-21iostat_ios06:22:511
236399743,9cyclictest6499-21ls05:13:041
236399743,8cyclictest22299-21cat06:57:171
236399743,55cyclictest0-21swapper/108:17:501
236399743,11cyclictest0-21swapper/109:27:101
236399743,10cyclictest0-21swapper/107:17:451
236399742,7cyclictest15226-21cat06:42:181
236399735,62cyclictest0-21swapper/106:12:381
236399734,61cyclictest0-21swapper/107:43:281
236399733,10cyclictest0-21swapper/105:37:591
236399724,61cyclictest0-21swapper/109:42:321
236399724,60cyclictest0-21swapper/105:22:221
236399723,8cyclictest13823-21munin-node07:47:481
236399723,62cyclictest17666-21sshd10:11:451
236399723,10cyclictest0-21swapper/110:32:411
236399723,10cyclictest0-21swapper/110:32:411
236399713,61cyclictest0-21swapper/107:57:411
236399713,61cyclictest0-21swapper/107:57:411
2363997116,48cyclictest20480-21grep09:07:431
2363997116,48cyclictest20480-21grep09:07:431
236399709,41cyclictest24224-21df_abs05:52:341
236399704,59cyclictest0-21swapper/105:07:211
236399703,61cyclictest0-21swapper/108:22:481
236399703,60cyclictest0-21swapper/108:47:591
236399703,10cyclictest0-21swapper/105:27:521
2363997012,52cyclictest29036-21df_abs06:02:431
236399694,10cyclictest16185-21munin-node07:52:361
236999634,52cyclictest1143-21snmpd09:42:443
235799633,11cyclictest0-21swapper/005:07:210
236999595,47cyclictest1199-21runrttasks07:38:253
236999584,46cyclictest18880-21yum-updatesd-he10:12:493
236999584,46cyclictest0-21swapper/307:12:433
236999584,46cyclictest0-21swapper/307:12:433
236999583,48cyclictest1143-21snmpd09:22:413
236999575,46cyclictest29787-21unixbench_singl08:17:533
2365995731,12cyclictest0-21swapper/208:57:452
70252563,45sleep20-21swapper/208:38:042
2365995639,10cyclictest9960-21unixbench_singl06:28:052
178052564,43sleep20-21swapper/209:02:392
236999554,44cyclictest0-21swapper/307:47:513
236999554,43cyclictest28654-21munin-node06:02:583
236999545,42cyclictest0-21swapper/307:08:213
236999544,43cyclictest26568-21yum-updatesd-he08:12:493
2365995423,26cyclictest0-21swapper/206:42:402
236999534,43cyclictest0-21swapper/308:42:483
236999534,42cyclictest4970-21munin-node05:12:473
236999534,42cyclictest0-21swapper/308:52:323
236999533,44cyclictest0-21swapper/305:57:483
236999533,43cyclictest0-21swapper/308:32:343
2369995319,9cyclictest22239-21yum-updatesd-he09:12:513
2369995314,32cyclictest1242-21munin-node09:37:383
2365995324,23cyclictest0-21swapper/207:57:502
2365995324,23cyclictest0-21swapper/207:57:502
2365995320,8cyclictest10365-21df_abs08:47:352
319132523,41sleep20-21swapper/207:17:362
236999524,42cyclictest0-21swapper/306:12:553
236999524,41cyclictest0-21swapper/309:47:293
236999524,41cyclictest0-21swapper/309:47:293
236999524,40cyclictest29923-21iostat_ios09:27:383
236999523,43cyclictest0-21swapper/307:32:523
236999523,42cyclictest6486-21munin-run07:32:163
2365995235,10cyclictest16436-21gltestperf10:07:352
11272524,40sleep31217-21missed_timers05:02:463
236999514,40cyclictest0-21swapper/309:32:343
236999514,40cyclictest0-21swapper/308:47:463
236999513,42cyclictest14858-21sed10:02:503
236599513,22cyclictest0-21swapper/208:52:302
2365995126,19cyclictest21806-21munin-node09:12:532
81662502,10sleep20-21swapper/209:51:322
81662502,10sleep20-21swapper/209:51:322
81522504,40sleep20-21swapper/207:32:552
54452503,10sleep30-21swapper/306:20:233
294802502,10sleep30-21swapper/310:36:183
294802502,10sleep30-21swapper/310:36:183
236999505,39cyclictest0-21swapper/306:28:073
236999505,38cyclictest24442-21df09:17:343
236999504,40cyclictest16562-21ntp_states06:42:493
236999504,40cyclictest0-21swapper/309:57:323
2369995035,9cyclictest24703-21head05:52:383
236999503,40cyclictest0-21swapper/308:22:373
2369995018,11cyclictest0-21swapper/306:57:523
2365995036,8cyclictest11692-21gltestperf09:57:362
2365995035,9cyclictest15135-21grep05:32:452
2365995035,10cyclictest12074-21grep07:42:392
2365995034,11cyclictest13941-21cut10:02:352
236599503,27cyclictest0-21swapper/206:37:172
236599502,41cyclictest20039-21munin-node06:52:572
2365995013,31cyclictest30358-21grep06:03:062
236999495,26cyclictest0-21swapper/306:42:133
236999494,40cyclictest0-21swapper/310:22:293
236999494,39cyclictest0-21swapper/305:47:393
236999494,39cyclictest0-21swapper/305:47:393
2369994934,9cyclictest22040-21sed10:17:503
2365994934,10cyclictest29738-21grep07:12:332
2365994934,10cyclictest29738-21grep07:12:332
2365994933,9cyclictest31833-21tune2fs09:32:352
103102492,40sleep20-21swapper/209:53:372
40192482,10sleep20-21swapper/207:26:062
24142482,10sleep30-21swapper/308:29:393
236999484,38cyclictest0-21swapper/308:07:433
236999484,37cyclictest0-21swapper/309:07:383
236999484,37cyclictest0-21swapper/309:07:383
236999484,27cyclictest0-21swapper/309:02:373
236999484,26cyclictest0-21swapper/308:07:153
236999483,39cyclictest0-21swapper/305:42:343
236599486,26cyclictest0-21swapper/205:27:412
2365994833,9cyclictest5057-21wc09:42:442
236599483,38cyclictest316-21munin-node08:27:402
2365994829,12cyclictest1810-21sed06:12:412
236599482,40cyclictest0-21swapper/205:21:422
2365994819,12cyclictest0-21swapper/205:12:492
236999475,37cyclictest0-21swapper/305:32:543
236999475,36cyclictest0-21swapper/308:57:303
236999474,27cyclictest0-21swapper/307:57:153
236999474,27cyclictest0-21swapper/307:17:533
236999474,25cyclictest0-21swapper/307:57:263
236999474,25cyclictest0-21swapper/307:57:263
236999474,24cyclictest0-21swapper/308:37:503
2369994732,10cyclictest9326-21sed09:52:383
2369994718,12cyclictest0-21swapper/305:11:593
2369994718,12cyclictest0-21swapper/305:11:593
2369994716,11cyclictest0-21swapper/306:37:133
2365994733,9cyclictest6472-21fschecks_time06:22:462
2365994732,9cyclictest29738-21grep08:17:532
2365994732,9cyclictest24694-21grep05:52:382
2365994732,10cyclictest28491-21proc_pri07:07:542
236599473,27cyclictest0-21swapper/206:38:022
236599473,26cyclictest26568-21yum-updatesd-he08:12:422
138022473,36sleep20-21swapper/207:47:182
236999464,36cyclictest0-21swapper/305:17:573
236999464,26cyclictest0-21swapper/307:45:493
236599467,23cyclictest0-21swapper/209:22:452
2365994632,9cyclictest17917-21grep07:53:042
2365994631,9cyclictest4295-21grep06:17:462
2365994631,9cyclictest21692-21tr05:47:322
2365994631,9cyclictest21692-21tr05:47:322
2365994630,9cyclictest10416-21ps07:37:512
236599463,26cyclictest0-21swapper/207:07:162
236599463,25cyclictest1143-21snmpd09:07:382
236599463,25cyclictest1143-21snmpd09:07:382
2365994625,7cyclictest264912cstates05:57:382
236599462,32cyclictest0-21swapper/208:12:002
2365994618,12cyclictest0-21swapper/209:27:382
2365994618,10cyclictest1199-21runrttasks05:37:552
2365994617,13cyclictest0-21swapper/209:37:472
2365994617,12cyclictest0-21swapper/210:17:462
175692462,10sleep30-21swapper/310:10:313
117842462,10sleep30-21swapper/305:26:553
26042452,10sleep20-21swapper/205:07:352
26042452,10sleep20-21swapper/205:07:352
236999454,25cyclictest0-21swapper/310:27:423
236999454,25cyclictest0-21swapper/306:57:163
236999454,25cyclictest0-21swapper/305:42:043
236599457,23cyclictest0-21swapper/206:47:432
236599454,24cyclictest0-21swapper/209:17:402
139232452,10sleep30-21swapper/305:28:573
236999444,35cyclictest0-21swapper/306:51:163
236999443,11cyclictest0-21swapper/307:22:483
2365994430,9cyclictest32059-21unixbench-2d08:22:522
236599443,35cyclictest0-21swapper/207:27:402
236599443,26cyclictest0-21swapper/207:02:172
2365994429,8cyclictest3592-21grep08:32:392
236599442,24cyclictest22678-21munin-node10:22:292
270452433,10sleep20-21swapper/210:30:292
236999434,34cyclictest0-21swapper/306:27:173
236999433,21cyclictest373-21runrttasks06:08:153
236599433,24cyclictest9868-21tr05:22:412
267362422,34sleep30-21swapper/307:05:003
236599423,33cyclictest0-21swapper/208:02:562
236599423,10cyclictest8020-21diskstats08:42:342
236599422,16cyclictest19161-21munin-node05:42:492
2365994214,13cyclictest0-21swapper/206:07:502
236599413,11cyclictest0-21swapper/210:32:272
236599413,11cyclictest0-21swapper/210:32:272
234599350,4rtkit-daemon2344-21rtkit-daemon05:02:442
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional