You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-03 - 18:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot6s.osadl.org (updated Wed Jun 03, 2026 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2828599983,7cyclictest107550irq/46-eth009:25:130
2828599965,72cyclictest0-21swapper/010:11:200
2828899954,49cyclictest0-21swapper/208:05:152
2828599954,84cyclictest0-21swapper/009:50:010
2828599953,77cyclictest4538-21ssh11:00:420
28285999411,66cyclictest0-21swapper/012:00:060
28285999333,43cyclictest19087-21grep10:35:050
28285999333,43cyclictest19087-21grep10:35:050
2828599933,82cyclictest0-21swapper/010:55:090
2828899925,30cyclictest1227-21runrttasks09:07:502
2828599924,26cyclictest0-21swapper/009:56:090
2828599924,26cyclictest0-21swapper/009:56:090
2828899914,41cyclictest0-21swapper/210:39:592
28288999139,11cyclictest1227-21runrttasks10:30:012
2828599914,79cyclictest0-21swapper/010:43:220
2828599913,24cyclictest0-21swapper/011:51:410
2828899904,14cyclictest0-21swapper/212:24:552
2828599905,14cyclictest0-21swapper/012:06:080
2828599904,78cyclictest0-21swapper/011:15:020
2828599904,78cyclictest0-21swapper/011:15:020
2828599904,71cyclictest0-21swapper/010:22:180
2828599904,25cyclictest11152-21proc_pri09:35:130
2828599903,22cyclictest21012-21diskmemload12:26:070
219499900,5rtkit-daemon2193-21rtkit-daemon10:17:021
2828599895,26cyclictest0-21swapper/010:05:170
2828599895,26cyclictest0-21swapper/009:43:500
2828599894,77cyclictest0-21swapper/007:34:150
2828599893,79cyclictest0-21swapper/011:06:080
28285998916,13cyclictest0-21swapper/009:17:150
28285998916,13cyclictest0-21swapper/009:17:150
2828999883,77cyclictest24355-21head12:19:513
2828899884,12cyclictest16389-21grep11:19:552
2828599884,29cyclictest0-21swapper/011:19:560
2828599884,23cyclictest0-21swapper/011:13:150
2828599883,78cyclictest0-21swapper/011:36:050
28285998827,54cyclictest32652-21unixbench_singl07:10:230
28285998827,54cyclictest32652-21unixbench_singl07:10:230
2828999874,75cyclictest0-21swapper/312:10:023
2828599874,75cyclictest0-21swapper/010:25:420
2828599874,28cyclictest0-21swapper/010:44:550
2828599874,25cyclictest0-21swapper/012:15:070
2828599873,77cyclictest0-21swapper/010:54:420
2828599873,23cyclictest0-21swapper/009:10:090
28285998723,57cyclictest0-21swapper/009:24:180
28285998718,12cyclictest0-21swapper/011:28:520
2828599864,74cyclictest0-21swapper/008:10:090
28285998628,52cyclictest0-21swapper/010:32:170
28285998621,12cyclictest0-21swapper/011:48:520
28285998618,12cyclictest0-21swapper/011:42:170
28285998616,62cyclictest23150-21cstates11:29:530
28285998614,12cyclictest0-21swapper/012:24:380
28285998612,12cyclictest0-21swapper/010:19:020
2828999856,71cyclictest4777-21perl11:49:573
2828999853,75cyclictest0-21swapper/310:47:433
28289998519,10cyclictest26812-21cpu09:59:563
2828599854,74cyclictest0-21swapper/012:14:190
2828599854,73cyclictest0-21swapper/009:47:480
2828599854,73cyclictest0-21swapper/009:34:050
2828599853,22cyclictest0-21swapper/009:06:150
28285998512,10cyclictest31358-21rm12:29:510
2828999846,20cyclictest0-21swapper/309:55:203
2828999846,20cyclictest0-21swapper/309:55:203
2828999843,74cyclictest0-21swapper/311:59:293
2828999843,74cyclictest0-21swapper/311:59:293
2828999843,74cyclictest0-21swapper/309:24:553
2828999843,73cyclictest1310-21runrttasks10:56:413
2828999843,73cyclictest0-21swapper/311:03:463
28289998420,50cyclictest12846-21kworker/3:111:31:563
28289998414,12cyclictest0-21swapper/310:44:173
28285998427,50cyclictest7701-21grep11:54:520
28285998427,50cyclictest7701-21grep11:54:520
2828999833,72cyclictest294382sleep310:50:083
28289998323,52cyclictest21053-21missed_timers09:50:083
2828599835,71cyclictest0-21swapper/009:59:480
28285998330,45cyclictest32338-21cpuspeed_turbos08:20:010
2828999824,70cyclictest0-21swapper/311:09:583
2828999823,71cyclictest0-21swapper/307:05:233
28289998228,48cyclictest1227-21runrttasks12:16:043
28289998222,43cyclictest31857-21unixbench_singl10:05:183
2828599824,33cyclictest0-21swapper/008:43:430
2828999814,24cyclictest0-21swapper/309:44:293
2828599814,69cyclictest0-21swapper/007:35:040
2828599814,23cyclictest16233-21awk08:54:460
2828599813,30cyclictest0-21swapper/008:45:010
2828999804,69cyclictest16947-21df09:44:573
2828999804,68cyclictest0-21swapper/311:44:553
28289998024,48cyclictest17267-21ssh11:20:063
28289998021,52cyclictest8571-21unixbench-2d08:35:193
28289998021,52cyclictest8571-21unixbench-2d08:35:193
28289998015,12cyclictest0-21swapper/310:35:113
28289998015,12cyclictest0-21swapper/310:35:113
2828599804,14cyclictest0-21swapper/007:27:170
2828999794,68cyclictest12348-21runrttasks07:35:273
2828999793,70cyclictest0-21swapper/311:36:583
2828999793,68cyclictest0-21swapper/309:35:033
28289997921,51cyclictest31811-21grep12:29:563
28289997917,44cyclictest0-21swapper/310:10:063
28289997917,10cyclictest4593-21telnet10:14:543
28289997913,10cyclictest1227-21runrttasks07:59:463
2828599794,66cyclictest0-21swapper/007:05:120
2828999785,10cyclictest29949-21df11:39:543
2828999783,68cyclictest16211-21ssh12:06:023
28289997822,7cyclictest21001-21crond09:09:393
28289997822,49cyclictest13349-21df07:40:033
28289997820,9cyclictest31632-21fschecks_time07:10:083
28289997820,9cyclictest31632-21fschecks_time07:10:083
2828599784,13cyclictest0-21swapper/009:04:000
2828599784,13cyclictest0-21swapper/008:05:170
2828599784,13cyclictest0-21swapper/007:20:230
28285997813,12cyclictest0-21swapper/008:30:170
2828999774,65cyclictest29742-21python09:15:143
2828999774,65cyclictest29742-21python09:15:143
2828999773,67cyclictest0-21swapper/309:14:133
28289997714,9cyclictest1547-21nscd11:05:153
2828599774,27cyclictest0-21swapper/007:40:220
2828599774,26cyclictest0-21swapper/007:53:480
2828599774,26cyclictest0-21swapper/007:53:480
2828599774,15cyclictest0-21swapper/008:35:200
2828599774,15cyclictest0-21swapper/008:35:200
28285997721,8cyclictest1227-21runrttasks07:14:580
2828999764,66cyclictest0-21swapper/310:25:083
2828999763,66cyclictest0-21swapper/310:19:543
28289997624,45cyclictest16113-21sort10:30:123
28289997615,13cyclictest0-21swapper/309:20:043
28287997622,27cyclictest0-21swapper/110:56:421
2828599764,26cyclictest0-21swapper/007:54:590
28285997615,12cyclictest0-21swapper/007:45:250
2828999753,65cyclictest21012-21diskmemload11:25:573
2828999753,22cyclictest0-21swapper/308:01:513
28289997516,8cyclictest18301-21fschecks_count07:50:063
28289997516,8cyclictest18301-21fschecks_count07:50:063
28289997516,11cyclictest0-21swapper/308:46:173
28285997522,8cyclictest17947-21switchtime08:55:170
28285997522,10cyclictest0-21swapper/008:00:010
2828999744,13cyclictest0-21swapper/311:15:323
2828999744,13cyclictest0-21swapper/311:15:323
2828999743,64cyclictest0-21swapper/312:00:023
28289997417,8cyclictest27888-21fschecks_count08:10:053
2828599743,30cyclictest0-21swapper/008:16:590
28285997420,10cyclictest3416-21sshd08:25:140
28285997413,10cyclictest28284-21systemd-cgroups07:04:500
2828999735,20cyclictest0-21swapper/307:15:233
2828999734,19cyclictest0-21swapper/308:20:013
2828999733,62cyclictest17767-21munin-node08:55:133
28289997321,9cyclictest30574-21hwlatdetect08:15:093
28289997318,9cyclictest1227-21runrttasks12:24:503
28289997316,9cyclictest3746-21timerwakeupswit08:25:203
28289997314,7cyclictest0-21swapper/309:30:163
28289997310,14cyclictest0-21swapper/308:49:593
2828999729,12cyclictest0-21swapper/307:25:053
2828999724,23cyclictest0-21swapper/309:04:453
2828999723,12cyclictest15385-21aten_r1power_fr07:45:013
28289997224,41cyclictest5948-21sendmail_mailst08:30:173
28289997127,38cyclictest8686-21grep07:30:053
261742710,5sleep1201ktimersoftd/108:05:151
28289997023,9cyclictest1227-21runrttasks07:24:063
28289997010,12cyclictest0-21swapper/308:05:073
219499680,5rtkit-daemon2193-21rtkit-daemon10:15:092
28289996710,12cyclictest0-21swapper/308:43:453
219499670,5rtkit-daemon2193-21rtkit-daemon11:47:212
28289996613,9cyclictest28297-21latency_hist07:04:503
28287996533,11cyclictest0-21swapper/111:34:521
2828899636,51cyclictest1227-21runrttasks09:56:402
2828899636,51cyclictest1227-21runrttasks09:56:402
28288996316,31cyclictest0-21swapper/211:03:252
2828799635,14cyclictest0-21swapper/112:15:541
113752628,8sleep20-21swapper/211:11:002
2828899615,13cyclictest0-21swapper/210:09:432
28288996119,23cyclictest0-21swapper/207:28:442
2828799615,13cyclictest25515-21perl10:44:591
2828799614,13cyclictest0-21swapper/110:04:431
219499610,5rtkit-daemon2193-21rtkit-daemon08:56:422
219499610,4rtkit-daemon2193-21rtkit-daemon09:13:042
2828899604,13cyclictest0-21swapper/212:29:592
2828799606,15cyclictest0-21swapper/111:00:081
2828799605,14cyclictest0-21swapper/109:49:231
2828799605,12cyclictest11888-21memory12:00:011
219499600,5rtkit-daemon2193-21rtkit-daemon09:50:332
2828899595,14cyclictest0-21swapper/211:05:172
2828899595,13cyclictest0-21swapper/211:24:422
2828899594,11cyclictest28278-21cyclictest09:39:592
28288995933,21cyclictest0-21swapper/211:43:132
2828899593,37cyclictest0-21swapper/212:14:512
2828899592,28cyclictest0-21swapper/210:29:322
28287995942,11cyclictest10997-21unixbench_multi08:40:201
150592594,48sleep10-21swapper/109:40:291
285502585,6sleep20-21swapper/211:37:082
2828899584,38cyclictest0-21swapper/210:44:552
2828899584,12cyclictest0-21swapper/210:00:022
28288995810,26cyclictest0-21swapper/209:30:032
28287995818,25cyclictest13978-21ntp_kernel_pll_11:15:061
28287995818,25cyclictest13978-21ntp_kernel_pll_11:15:061
203822583,11sleep10-21swapper/110:37:171
203822583,11sleep10-21swapper/110:37:171
2828899577,41cyclictest0-21swapper/210:49:582
2828899573,35cyclictest0-21swapper/210:38:232
2828899573,35cyclictest0-21swapper/210:38:232
28287995737,13cyclictest17836-21tr12:09:511
59472562,10sleep20-21swapper/209:29:092
316362563,46sleep10-21swapper/108:17:451
314042563,10sleep131528-21df_abs12:29:521
2828899565,34cyclictest0-21swapper/209:35:332
2828899565,34cyclictest0-21swapper/209:20:392
2828899565,30cyclictest0-21swapper/210:11:522
28288995614,24cyclictest20222-21ssh12:13:182
2828799564,34cyclictest1124-21snmpd10:21:251
2828899555,31cyclictest0-21swapper/211:54:522
2828899555,31cyclictest0-21swapper/211:54:522
2828899554,32cyclictest0-21swapper/209:14:582
2828899554,32cyclictest0-21swapper/209:14:582
2828899554,13cyclictest0-21swapper/212:05:262
28288995526,13cyclictest0-21swapper/209:49:222
28288995522,14cyclictest0-21swapper/211:16:242
28288995522,14cyclictest0-21swapper/211:16:242
2828799556,39cyclictest22768-21ssh11:29:491
2828799555,30cyclictest0-21swapper/111:47:071
2828799555,30cyclictest0-21swapper/110:10:031
2828799555,29cyclictest1227-21runrttasks07:48:471
2828799553,21cyclictest31396-21taskset11:40:141
28287995511,16cyclictest0-21swapper/108:59:581
263672553,11sleep20-21swapper/212:21:112
2828899546,30cyclictest0-21swapper/208:35:042
2828899546,30cyclictest0-21swapper/208:35:042
2828899544,29cyclictest0-21swapper/210:19:562
28288995422,13cyclictest0-21swapper/211:34:402
2828799545,30cyclictest0-21swapper/109:24:041
2828799545,27cyclictest0-21swapper/108:19:591
28287995431,16cyclictest0-21swapper/111:54:491
28287995431,16cyclictest0-21swapper/111:54:491
28287995422,12cyclictest0-21swapper/107:29:481
2828899535,31cyclictest0-21swapper/210:55:122
2828899535,29cyclictest0-21swapper/208:10:042
2828899535,29cyclictest0-21swapper/207:40:022
2828899534,43cyclictest0-21swapper/207:35:062
28288995333,12cyclictest30330-21grep08:15:062
2828799535,16cyclictest0-21swapper/111:27:531
2828799535,14cyclictest0-21swapper/110:28:231
2828799534,29cyclictest0-21swapper/109:59:431
2828799534,29cyclictest0-21swapper/109:59:431
2828799534,24cyclictest0-21swapper/107:30:011
28287995334,12cyclictest22165-21tail10:40:001
28287995322,13cyclictest0-21swapper/109:14:551
28287995322,13cyclictest0-21swapper/109:14:551
28287995320,27cyclictest25046-21cpuspeed09:10:001
28287995316,21cyclictest0-21swapper/111:09:511
219499530,5rtkit-daemon2193-21rtkit-daemon07:45:002
54862522,11sleep20-21swapper/207:24:222
2828899525,30cyclictest0-21swapper/207:10:202
2828899525,30cyclictest0-21swapper/207:10:202
2828899525,29cyclictest0-21swapper/211:50:002
2828899525,28cyclictest0-21swapper/208:50:032
2828899525,28cyclictest0-21swapper/208:40:062
28288995233,10cyclictest2501-21grep08:25:012
28288995221,11cyclictest0-21swapper/207:15:012
2828799525,38cyclictest0-21swapper/110:09:441
2828799525,29cyclictest0-21swapper/111:20:111
2828799525,29cyclictest0-21swapper/109:34:561
2828799525,14cyclictest0-21swapper/112:09:301
2828799524,30cyclictest0-21swapper/110:30:131
2828799524,28cyclictest0-21swapper/111:05:031
2828799524,11cyclictest20310-21df09:49:581
2828799523,28cyclictest0-21swapper/109:30:131
28287995220,11cyclictest21839-21ssh09:05:001
222152523,11sleep20-21swapper/207:59:252
208932522,10sleep20-21swapper/209:03:252
126552523,43sleep10-21swapper/107:39:281
293542513,8sleep229426-21grep07:05:112
2828899514,28cyclictest0-21swapper/207:33:452
28288995132,10cyclictest11086-21grep11:59:512
2828799516,28cyclictest0-21swapper/109:29:281
2828799515,28cyclictest0-21swapper/108:10:171
2828799515,28cyclictest0-21swapper/107:40:121
2828799515,28cyclictest0-21swapper/107:15:061
2828799515,27cyclictest0-21swapper/111:54:391
2828799514,29cyclictest0-21swapper/108:55:111
2828799514,28cyclictest0-21swapper/112:19:581
28287995119,13cyclictest0-21swapper/108:34:461
219492513,11sleep10-21swapper/107:55:571
219499510,5rtkit-daemon2193-21rtkit-daemon08:20:062
2828899505,35cyclictest0-21swapper/208:00:152
28288995031,13cyclictest0-21swapper/207:50:062
28288995031,13cyclictest0-21swapper/207:50:062
28288995020,12cyclictest0-21swapper/208:49:072
2828799505,26cyclictest29123-21iostat_ios10:50:021
2828799504,27cyclictest0-21swapper/108:54:471
28287995019,14cyclictest0-21swapper/112:25:341
2828899494,26cyclictest0-21swapper/208:33:552
2828799494,27cyclictest0-21swapper/107:23:441
2828799494,27cyclictest0-21swapper/107:05:021
28287994932,10cyclictest19355-21unixbench-2d07:50:211
28287994932,10cyclictest19355-21unixbench-2d07:50:211
28287994910,21cyclictest0-21swapper/108:24:591
74752483,10sleep17483-21diskstats08:35:031
74752483,10sleep17483-21diskstats08:35:031
2828799484,27cyclictest0-21swapper/108:48:581
276252474,6sleep20-21swapper/207:00:172
2828799466,24cyclictest0-21swapper/107:10:011
2828799466,24cyclictest0-21swapper/107:10:011
2828799454,32cyclictest24420-21runrttasks08:02:061
28287994514,12cyclictest0-21swapper/107:04:501
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional