You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-06 - 22:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot6s.osadl.org (updated Mon Apr 06, 2026 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
36852920,8sleep32122799cyclictest09:32:263
21227999222,12cyclictest0-21swapper/309:07:573
21227999131,9cyclictest19404-21fschecks_count10:42:433
21227999029,10cyclictest6206-21gltestperf10:22:463
21227999027,13cyclictest0-21swapper/309:52:283
21227999027,13cyclictest0-21swapper/309:22:433
21227999023,12cyclictest0-21swapper/308:07:433
21227998823,12cyclictest0-21swapper/310:33:013
21227998823,12cyclictest0-21swapper/308:47:583
21227998821,12cyclictest0-21swapper/308:33:123
21227998616,27cyclictest3518-21gltestperf08:42:493
226722850,10sleep32122799cyclictest06:32:473
21227998534,8cyclictest31763-21sed10:12:473
21227998531,9cyclictest9879-21missed_timers10:27:523
21227998530,10cyclictest0-21swapper/310:51:413
2122799843,46cyclictest0-21swapper/310:05:283
21227998322,12cyclictest0-21swapper/309:57:443
21227998221,12cyclictest0-21swapper/306:12:553
21227998121,13cyclictest0-21swapper/310:07:443
21227998110,14cyclictest0-21swapper/308:17:313
2122799804,13cyclictest0-21swapper/309:42:263
2122799773,11cyclictest4520-21unixbench_singl07:53:043
2122799754,14cyclictest0-21swapper/305:57:493
21227997532,10cyclictest16482-21sshd07:24:423
21227997527,7cyclictest17742-21cat06:22:373
21227997527,7cyclictest17742-21cat06:22:373
2122799752,22cyclictest0-21swapper/309:54:483
2122799744,14cyclictest0-21swapper/308:17:483
2122799744,14cyclictest0-21swapper/308:17:483
2122599745,61cyclictest0-21swapper/110:30:191
2122799734,15cyclictest0-21swapper/309:14:573
2122799725,14cyclictest0-21swapper/309:32:433
2122799714,15cyclictest0-21swapper/309:02:373
2122799714,15cyclictest0-21swapper/309:02:373
2122599717,58cyclictest0-21swapper/108:45:181
2122599716,58cyclictest0-21swapper/108:07:411
2122599715,59cyclictest0-21swapper/109:27:461
2122599706,57cyclictest0-21swapper/107:48:221
233382699,53sleep20-21swapper/210:00:282
2122799696,13cyclictest0-21swapper/307:43:083
2122799696,13cyclictest0-21swapper/307:43:083
2122599695,57cyclictest0-21swapper/110:23:131
2122599694,57cyclictest0-21swapper/109:47:491
2122599694,13cyclictest0-21swapper/110:07:541
2122599693,59cyclictest0-21swapper/110:48:551
2122599685,56cyclictest0-21swapper/109:47:101
2122599685,56cyclictest0-21swapper/108:52:451
2122599685,55cyclictest10030-21diskstats08:02:481
2122599684,57cyclictest0-21swapper/110:13:061
2122399684,57cyclictest19185-21df10:42:390
2475626716,8sleep124821-21expr10:02:441
21226996743,8cyclictest9126-21idleruntime-cro08:52:302
21226996741,12cyclictest0-21swapper/209:07:062
21226996741,12cyclictest0-21swapper/209:07:062
2122599676,54cyclictest0-21swapper/108:34:271
2122599674,57cyclictest0-21swapper/110:17:451
2122699664,55cyclictest0-21swapper/210:47:392
2122599665,54cyclictest0-21swapper/110:39:331
2122599665,54cyclictest0-21swapper/108:14:421
2122599665,53cyclictest1419-21runrttasks09:14:211
2122599664,55cyclictest0-21swapper/108:50:301
21226996541,9cyclictest19438-21grep07:28:062
21226996540,12cyclictest0-21swapper/210:08:042
21226996539,12cyclictest0-21swapper/209:07:592
2122399653,54cyclictest0-21swapper/010:35:050
21226996438,7cyclictest24819-21ssh07:37:462
21226996438,13cyclictest0-21swapper/208:13:082
21226996437,14cyclictest0-21swapper/209:37:592
21226996434,15cyclictest0-21swapper/210:04:352
2122599644,53cyclictest0-21swapper/110:47:261
2122399643,55cyclictest0-21swapper/007:58:550
21226996338,12cyclictest0-21swapper/210:22:372
21226996337,12cyclictest0-21swapper/208:37:422
21226996337,12cyclictest0-21swapper/207:55:442
21226996336,13cyclictest0-21swapper/209:47:142
21226996336,13cyclictest0-21swapper/206:37:522
21226996336,11cyclictest0-21swapper/210:42:572
2122599634,53cyclictest31916-21kworker/1:108:58:271
2122399633,54cyclictest7968-21ssh08:48:180
2122399633,54cyclictest309002sleep010:12:380
2122399633,54cyclictest11311-21vmstat08:03:080
2122399633,53cyclictest0-21swapper/008:57:450
264742623,22sleep026604-21tr08:27:460
2122699624,52cyclictest0-21swapper/210:12:412
21226996239,7cyclictest14341-21ssh10:34:352
21226996237,12cyclictest0-21swapper/208:23:022
21226996237,12cyclictest0-21swapper/208:23:022
21226996237,12cyclictest0-21swapper/206:47:512
21226996236,13cyclictest0-21swapper/208:54:572
21226996236,12cyclictest0-21swapper/208:18:062
21226996236,12cyclictest0-21swapper/208:18:062
21226996234,13cyclictest0-21swapper/208:07:112
21226996233,14cyclictest0-21swapper/209:36:102
2122699623,53cyclictest0-21swapper/208:44:592
2122599625,51cyclictest0-21swapper/109:10:191
2122599625,50cyclictest0-21swapper/109:57:461
2122599625,50cyclictest0-21swapper/107:38:401
2122699614,51cyclictest0-21swapper/209:47:502
2122699614,51cyclictest0-21swapper/209:12:592
2122699614,51cyclictest0-21swapper/208:08:012
21226996136,12cyclictest0-21swapper/208:58:042
21226996135,13cyclictest0-21swapper/207:07:472
21226996135,12cyclictest0-21swapper/209:55:482
21226996134,13cyclictest0-21swapper/209:17:402
2122699613,48cyclictest0-21swapper/207:32:502
2122699604,50cyclictest0-21swapper/210:18:002
21226996034,12cyclictest0-21swapper/209:23:442
21226996033,12cyclictest1681-21nscd06:09:212
21226996032,15cyclictest0-21swapper/209:30:252
2122599604,49cyclictest0-21swapper/109:23:221
2122599603,50cyclictest0-21swapper/107:52:471
2122399604,49cyclictest0-21swapper/009:47:390
2122399604,48cyclictest0-21swapper/008:33:030
21223996036,8cyclictest19914-21ssh09:07:450
21223996035,10cyclictest3572-21gltestperf08:42:500
2122399603,51cyclictest0-21swapper/010:28:530
2122399603,51cyclictest0-21swapper/009:41:500
2122399603,48cyclictest0-21swapper/006:13:040
2122399602,52cyclictest11635-21kworker/0:207:42:510
2122399602,52cyclictest11635-21kworker/0:207:42:510
2122399602,50cyclictest0-21swapper/010:07:570
188392603,13sleep10-21swapper/106:22:591
2122699594,28cyclictest0-21swapper/205:37:592
2122699594,28cyclictest0-21swapper/205:37:592
21226995936,10cyclictest0-21swapper/206:33:032
21226995934,11cyclictest0-21swapper/210:42:262
21226995934,11cyclictest0-21swapper/208:35:182
21226995934,11cyclictest0-21swapper/208:28:382
21226995934,11cyclictest0-21swapper/207:47:522
21226995933,11cyclictest0-21swapper/207:22:472
2122699593,50cyclictest0-21swapper/208:02:262
2122599595,47cyclictest0-21swapper/108:19:421
2122599595,47cyclictest0-21swapper/108:19:421
2122599594,49cyclictest0-21swapper/106:37:471
2122599593,34cyclictest0-21swapper/109:20:531
21225995913,14cyclictest0-21swapper/108:26:101
21225995913,14cyclictest0-21swapper/108:26:101
2122399594,48cyclictest0-21swapper/010:47:370
2122399594,48cyclictest0-21swapper/008:38:050
21223995933,8cyclictest24161-21ssh09:12:570
21223995932,12cyclictest0-21swapper/006:53:020
2122399593,49cyclictest0-21swapper/009:42:430
2122399593,49cyclictest0-21swapper/009:32:580
21223995926,13cyclictest0-21swapper/009:23:130
282552582,49sleep00-21swapper/005:36:140
2122699584,48cyclictest0-21swapper/206:22:572
21226995831,11cyclictest8676-21munin-node10:28:022
2122699583,50cyclictest0-21swapper/207:42:452
2122699583,50cyclictest0-21swapper/207:42:452
21226995828,14cyclictest0-21swapper/206:27:532
2122599585,47cyclictest0-21swapper/109:56:341
2122599584,48cyclictest0-21swapper/107:36:561
2122599584,47cyclictest0-21swapper/108:27:441
2122599584,39cyclictest0-21swapper/107:22:331
2122599584,13cyclictest0-21swapper/106:32:371
2122599583,26cyclictest0-21swapper/107:57:451
2122599583,22cyclictest0-21swapper/106:32:471
21225995815,12cyclictest0-21swapper/109:40:551
21225995811,10cyclictest977-21ntpd09:32:551
2122399583,50cyclictest0-21swapper/008:27:320
2122399583,50cyclictest0-21swapper/008:27:320
21223995828,13cyclictest0-21swapper/009:27:550
2122699573,49cyclictest0-21swapper/206:13:092
21226995727,12cyclictest0-21swapper/207:19:482
2122599576,29cyclictest0-21swapper/108:41:151
2122599576,28cyclictest0-21swapper/106:52:461
2122599575,46cyclictest0-21swapper/107:23:411
2122599575,45cyclictest0-21swapper/105:37:471
2122599575,45cyclictest0-21swapper/105:37:471
2122599574,25cyclictest0-21swapper/107:31:421
2122599573,23cyclictest9373-21memory06:03:011
21225995715,11cyclictest13952-21diskmemload07:45:571
21225995715,11cyclictest13952-21diskmemload07:45:571
21223995735,9cyclictest3686-21ssh07:52:510
2122399573,50cyclictest0-21swapper/010:06:160
2122399573,47cyclictest0-21swapper/010:17:470
21223995728,12cyclictest0-21swapper/009:57:450
112612572,47sleep00-21swapper/007:13:400
21226995618,15cyclictest0-21swapper/206:43:012
2122599565,45cyclictest0-21swapper/110:32:401
2122599563,26cyclictest0-21swapper/106:17:511
2122599563,26cyclictest0-21swapper/106:17:511
21225995617,33cyclictest16713-21entropy09:02:471
21225995617,33cyclictest16713-21entropy09:02:471
21225995617,10cyclictest9518-21aten2.4-expect07:12:451
21225995617,10cyclictest30989-21munin-node05:43:051
21225995616,10cyclictest6283-21df05:57:511
21223995632,9cyclictest15471-21aten_r1power_fr10:37:380
2122399563,49cyclictest0-21swapper/010:26:030
2122399563,45cyclictest0-21swapper/005:27:520
21223995629,12cyclictest0-21swapper/008:54:590
21223995628,11cyclictest0-21swapper/006:17:530
21223995628,11cyclictest0-21swapper/006:17:530
21223995626,14cyclictest0-21swapper/007:32:450
21223995625,12cyclictest0-21swapper/007:48:220
2122399562,49cyclictest0-21swapper/007:28:320
30992553,9sleep00-21swapper/005:50:530
2122699555,43cyclictest0-21swapper/205:32:492
21226995527,13cyclictest0-21swapper/205:42:522
21226995519,13cyclictest0-21swapper/206:57:532
21226995519,13cyclictest0-21swapper/206:57:532
2122599554,45cyclictest0-21swapper/105:47:481
2122599554,27cyclictest0-21swapper/106:13:031
21225995517,10cyclictest29936-21cpu06:47:461
21225995516,10cyclictest10656-21aten2.4-expect06:07:471
21225995515,13cyclictest0-21swapper/105:53:021
21225995515,12cyclictest0-21swapper/107:08:031
2122399554,47cyclictest0-21swapper/009:06:510
2122399554,47cyclictest0-21swapper/009:06:510
2122399554,45cyclictest0-21swapper/009:20:100
21223995533,8cyclictest19801-21ssh08:17:450
21223995533,8cyclictest19801-21ssh08:17:450
21223995531,9cyclictest12830-21telnet08:07:430
2122399553,47cyclictest0-21swapper/007:23:000
2122399553,46cyclictest0-21swapper/006:32:590
21223995528,12cyclictest0-21swapper/007:41:080
2122699543,8cyclictest9664-21find06:03:062
2122699543,43cyclictest5879-21aten_r1power_cu05:57:482
21226995428,12cyclictest0-21swapper/206:54:592
21226995415,13cyclictest0-21swapper/206:21:412
21226995415,13cyclictest0-21swapper/206:21:412
2122599544,27cyclictest0-21swapper/105:31:301
21225995416,13cyclictest0-21swapper/107:02:451
21225995416,13cyclictest0-21swapper/107:02:451
21225995416,10cyclictest2122-21munin-node06:58:021
21225995416,10cyclictest2122-21munin-node06:58:021
21225995410,22cyclictest0-21swapper/105:36:301
21223995426,11cyclictest0-21swapper/008:13:330
2122399542,47cyclictest0-21swapper/009:56:180
232662533,9sleep00-21swapper/005:23:240
21226995328,12cyclictest0-21swapper/207:15:422
2122399533,45cyclictest0-21swapper/006:43:060
2122399533,44cyclictest0-21swapper/007:17:460
2122399533,43cyclictest0-21swapper/006:01:340
21223995327,12cyclictest0-21swapper/006:42:350
248599520,5rtkit-daemon2484-21rtkit-daemon05:23:373
21226995228,12cyclictest0-21swapper/205:28:152
21226995222,13cyclictest0-21swapper/207:06:472
21226995222,13cyclictest0-21swapper/207:06:472
21226995219,15cyclictest0-21swapper/205:52:572
21226995216,12cyclictest0-21swapper/205:22:592
2122399523,44cyclictest0-21swapper/005:37:520
2122399523,44cyclictest0-21swapper/005:37:520
21223995225,10cyclictest0-21swapper/006:57:460
21223995225,10cyclictest0-21swapper/006:57:460
125692523,10sleep00-21swapper/006:09:520
235402513,42sleep10-21swapper/105:27:001
21227995130,8cyclictest23830-21munin-node05:27:593
2122699515,24cyclictest0-21swapper/205:47:502
2122399512,41cyclictest7366-21cpuspeed_turbos07:07:490
21223995119,12cyclictest0-21swapper/005:43:020
23532502,42sleep30-21swapper/308:42:283
21227995023,13cyclictest0-21swapper/308:27:323
21227995023,13cyclictest0-21swapper/308:27:323
21227995022,13cyclictest0-21swapper/307:17:343
2122399503,31cyclictest0-21swapper/006:47:590
2122399502,42cyclictest0-21swapper/007:02:440
2122399502,42cyclictest0-21swapper/007:02:440
2122799494,12cyclictest0-21swapper/309:22:293
21227994934,9cyclictest552-21gltestperf06:52:543
21227994934,9cyclictest12400-21ssh08:07:043
21227994927,17cyclictest22355-21latency_hist06:32:363
21227994926,9cyclictest3747-21tail10:18:003
21227994919,10cyclictest17000-21expr10:37:593
21223994930,7cyclictest4794-21proc_pri05:53:060
293262483,9sleep10-21swapper/106:45:111
2122799483,39cyclictest0-21swapper/308:31:303
21227994820,12cyclictest0-21swapper/309:02:303
2122399484,34cyclictest0-21swapper/006:29:350
2122399483,27cyclictest0-21swapper/006:07:000
2122799477,11cyclictest0-21swapper/307:40:133
21227994733,9cyclictest1287-21snmpd07:11:483
21227994731,10cyclictest30344-21fschecks_count06:47:513
2122799473,39cyclictest0-21swapper/307:52:053
2122799473,38cyclictest0-21swapper/309:43:223
21227994720,13cyclictest0-21swapper/308:57:303
21223994725,11cyclictest0-21swapper/006:26:410
2122799463,37cyclictest0-21swapper/307:32:553
21227994627,7cyclictest6715-21yum-updatesd-he07:06:553
21227994627,7cyclictest6715-21yum-updatesd-he07:06:553
21227994626,8cyclictest8642-21ssh08:00:403
21227994624,8cyclictest11570-21basename06:07:563
211122464,35sleep00-21swapper/005:21:390
29882455,34sleep30-21swapper/305:49:273
2122799454,12cyclictest0-21swapper/306:45:503
2122699454,27cyclictest0-21swapper/205:22:392
185582450,14sleep32122799cyclictest07:27:543
21227994418,12cyclictest0-21swapper/305:38:013
21227994418,12cyclictest0-21swapper/305:38:013
211252444,6sleep10-21swapper/105:21:481
118022442,35sleep30-21swapper/307:17:463
21227994327,10cyclictest9050-21gltestperf06:02:563
21227994317,9cyclictest31862-21gltestperf05:42:563
2122799423,33cyclictest26274-21aten_r1power_en05:32:483
2122799423,33cyclictest26258-21ntp_states06:38:023
2122799423,11cyclictest0-21swapper/305:52:483
2122799413,33cyclictest0-21swapper/306:58:023
2122799413,33cyclictest0-21swapper/306:58:023
21227994114,12cyclictest0-21swapper/306:22:553
2122799373,28cyclictest21198-21latency_hist05:22:383
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional