You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-24 - 18:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot6s.osadl.org (updated Sun May 24, 2026 12:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
287909910033,9cyclictest7173-21latency_hist06:07:291
2879299993,45cyclictest1643-21aten_r1power_fr07:42:363
28787999935,57cyclictest0-21swapper/006:42:440
2879099985,46cyclictest0-21swapper/109:07:461
2879099985,46cyclictest0-21swapper/109:07:461
28787999832,8cyclictest15363-21cpuspeed08:02:380
28792999636,9cyclictest18788-21ssh08:07:383
28787999635,54cyclictest0-21swapper/008:22:190
2879299953,45cyclictest4757-21chrt06:57:533
28790999532,9cyclictest6775-21munin-node07:47:581
28787999542,46cyclictest30109-21sh07:36:410
2879299943,49cyclictest19599-21timerandwakeup09:42:533
28790999422,29cyclictest17454-21gltestperf08:52:431
28787999432,12cyclictest0-21swapper/009:47:440
28787999338,48cyclictest0-21swapper/008:07:500
2879299924,52cyclictest0-21swapper/306:52:483
28792999224,11cyclictest19361-21munin-node06:32:573
2879099925,27cyclictest17531-21runrttasks08:04:521
2878799927,36cyclictest0-21swapper/006:39:410
2878799925,36cyclictest0-21swapper/008:12:400
2878799924,40cyclictest0-21swapper/009:52:350
2878799924,40cyclictest0-21swapper/009:52:350
28787999234,51cyclictest0-21swapper/006:47:490
293852910,3sleep226-21rcuc/208:22:422
28787999134,12cyclictest0-21swapper/007:21:000
2879099905,38cyclictest1419-21runrttasks07:42:461
236392900,5sleep023644-21cpuspeed_turbos09:02:350
28787998937,8cyclictest22988-21munin-node05:32:420
2878799893,36cyclictest0-21swapper/009:17:230
2879299884,42cyclictest4134-21iostat08:32:473
2878799884,35cyclictest0-21swapper/006:23:450
2878799883,37cyclictest0-21swapper/007:27:480
2878799874,37cyclictest0-21swapper/008:52:350
2878799874,28cyclictest0-21swapper/009:27:220
28787998735,13cyclictest0-21swapper/009:19:050
2878799869,32cyclictest0-21swapper/007:37:390
2878799867,35cyclictest0-21swapper/010:02:340
2878799865,11cyclictest0-21swapper/006:57:510
28787998638,42cyclictest0-21swapper/007:26:300
28787998633,13cyclictest0-21swapper/008:28:580
28787998633,13cyclictest0-21swapper/008:28:580
2878799863,36cyclictest0-21swapper/009:32:370
28787998624,8cyclictest16829-21sh07:17:260
28787998619,12cyclictest0-21swapper/009:27:450
2878799854,36cyclictest0-21swapper/008:57:400
28787998527,12cyclictest0-21swapper/006:17:470
62772830,5sleep16275-21grep09:22:561
2878799834,35cyclictest0-21swapper/007:42:300
28787998333,10cyclictest0-21swapper/008:24:060
28787998326,14cyclictest0-21swapper/007:02:490
28787998322,12cyclictest0-21swapper/005:47:550
28787998319,9cyclictest17965-21latency_hist05:22:320
28787998219,13cyclictest0-21swapper/008:37:580
2878799805,36cyclictest0-21swapper/008:01:000
28787998033,10cyclictest0-21swapper/009:09:130
28787998033,10cyclictest0-21swapper/009:09:130
28787998023,20cyclictest0-21swapper/007:56:010
28787998022,11cyclictest0-21swapper/009:46:180
28787997932,10cyclictest0-21swapper/007:47:340
28787997829,9cyclictest0-21swapper/006:55:120
2878799778,28cyclictest0-21swapper/005:27:360
28787997721,7cyclictest11947-21latency_hist06:17:290
28787997521,11cyclictest0-21swapper/005:59:010
28787997432,10cyclictest0-21swapper/007:11:200
28787997432,10cyclictest0-21swapper/007:11:200
15922740,6sleep3361ktimersoftd/305:53:023
2878799724,15cyclictest0-21swapper/008:47:510
28787997232,9cyclictest0-21swapper/009:38:140
2878799723,16cyclictest0-21swapper/008:42:480
2661027011,48sleep00-21swapper/004:32:440
248599680,5rtkit-daemon2484-21rtkit-daemon07:54:323
248599680,5rtkit-daemon2484-21rtkit-daemon07:51:402
311622620,3sleep331165-21ssh08:25:263
2879199622,22cyclictest0-21swapper/209:39:132
28790996231,14cyclictest0-21swapper/107:22:511
248599620,5rtkit-daemon2484-21rtkit-daemon06:01:142
2879299613,51cyclictest5818-21fschecks_time07:47:433
2879299613,51cyclictest0-21swapper/309:12:423
28790996136,10cyclictest10964-21systemd-cgroups09:32:211
248599610,13rtkit-daemon2484-21rtkit-daemon07:22:562
2879199604,34cyclictest0-21swapper/209:18:192
2879099605,47cyclictest29669-21irqstats08:22:471
242742603,50sleep20-21swapper/205:32:592
228202605,47sleep10-21swapper/108:12:461
2879099594,47cyclictest0-21swapper/109:37:261
2879299583,48cyclictest0-21swapper/308:52:363
28791995821,15cyclictest0-21swapper/208:47:462
28790995830,12cyclictest0-21swapper/107:02:191
28790995829,14cyclictest0-21swapper/107:32:491
28790995829,13cyclictest0-21swapper/106:39:331
28791995723,26cyclictest0-21swapper/207:00:412
2879099579,42cyclictest16285-21awk08:52:221
2879099574,46cyclictest29904-21ls06:47:531
28790995725,11cyclictest0-21swapper/106:57:281
2879299564,45cyclictest0-21swapper/306:01:443
28791995634,15cyclictest22570-21tune2fs06:37:462
2879199563,47cyclictest0-21swapper/209:02:372
2879099564,46cyclictest0-21swapper/109:48:141
2879099564,46cyclictest0-21swapper/109:43:591
2879099564,46cyclictest0-21swapper/107:07:431
2879099564,46cyclictest0-21swapper/107:07:431
2879099564,45cyclictest26089-21proc_pri09:52:511
2879099564,45cyclictest26089-21proc_pri09:52:511
28790995627,13cyclictest0-21swapper/108:42:401
28790995626,12cyclictest0-21swapper/107:57:381
28787995630,12cyclictest0-21swapper/005:22:460
248599560,6rtkit-daemon2484-21rtkit-daemon04:47:323
248599560,5rtkit-daemon2484-21rtkit-daemon06:42:582
57682553,45sleep20-21swapper/208:35:582
28792995520,14cyclictest0-21swapper/309:10:133
28792995520,14cyclictest0-21swapper/309:10:133
28791995535,12cyclictest28508-21grep05:42:492
2879199552,23cyclictest0-21swapper/209:23:392
2879099554,45cyclictest0-21swapper/107:17:281
28790995527,12cyclictest0-21swapper/108:32:301
28790995527,12cyclictest0-21swapper/108:28:321
28790995527,12cyclictest0-21swapper/108:28:321
28790995525,8cyclictest8251-21ssh08:37:581
272652554,43sleep10-21swapper/106:43:451
248599550,5rtkit-daemon2484-21rtkit-daemon08:12:392
248599550,5rtkit-daemon2484-21rtkit-daemon07:52:472
248599550,4rtkit-daemon2484-21rtkit-daemon05:25:082
2879299545,29cyclictest0-21swapper/309:47:523
2879299545,29cyclictest0-21swapper/308:12:463
2879299544,25cyclictest377-21ssh07:39:203
2879299543,46cyclictest0-21swapper/307:23:523
2879199544,12cyclictest0-21swapper/207:37:402
2879099545,43cyclictest0-21swapper/109:58:081
2879099544,46cyclictest0-21swapper/107:07:241
2879099543,26cyclictest0-21swapper/105:37:571
28790995427,12cyclictest0-21swapper/104:41:571
28790995426,13cyclictest0-21swapper/110:05:221
28790995426,12cyclictest0-21swapper/109:16:061
28790995426,12cyclictest0-21swapper/107:14:001
28790995425,13cyclictest0-21swapper/107:40:311
28790995425,12cyclictest0-21swapper/107:54:111
28787995436,12cyclictest0-21swapper/008:32:470
248599540,5rtkit-daemon2484-21rtkit-daemon09:27:442
248599540,5rtkit-daemon2484-21rtkit-daemon07:32:482
248599540,4rtkit-daemon2484-21rtkit-daemon08:57:072
324652532,10sleep20-21swapper/210:02:412
2879299535,41cyclictest0-21swapper/308:37:423
2879299534,7cyclictest27173-21sh07:32:283
2879299534,41cyclictest0-21swapper/307:17:373
2879299533,45cyclictest0-21swapper/305:22:553
2879299533,44cyclictest0-21swapper/309:57:313
2879299533,29cyclictest0-21swapper/308:27:373
2879299533,29cyclictest0-21swapper/308:27:373
28792995315,13cyclictest121850irq/46-eth008:42:443
2879199538,15cyclictest0-21swapper/209:52:392
2879199538,15cyclictest0-21swapper/209:52:392
2879199534,28cyclictest0-21swapper/209:42:452
2879199534,27cyclictest0-21swapper/207:17:372
2879199534,12cyclictest0-21swapper/207:07:382
2879199534,12cyclictest0-21swapper/207:07:382
28791995335,11cyclictest26058-21grep05:37:482
28791995335,11cyclictest21126-21grep06:33:022
28790995328,11cyclictest0-21swapper/105:07:531
28790995328,11cyclictest0-21swapper/105:07:531
28790995328,10cyclictest23716-21df09:02:361
28790995326,12cyclictest0-21swapper/109:32:541
28790995325,13cyclictest0-21swapper/108:17:411
28790995325,13cyclictest0-21swapper/105:02:441
28790995325,12cyclictest0-21swapper/108:57:591
28790995324,12cyclictest0-21swapper/108:07:581
248599530,5rtkit-daemon2484-21rtkit-daemon08:57:522
248599530,5rtkit-daemon2484-21rtkit-daemon08:41:082
248599530,5rtkit-daemon2484-21rtkit-daemon08:07:472
248599530,5rtkit-daemon2484-21rtkit-daemon07:27:412
248599530,5rtkit-daemon2484-21rtkit-daemon06:28:062
248599530,5rtkit-daemon2484-21rtkit-daemon06:28:062
248599530,5rtkit-daemon2484-21rtkit-daemon05:07:462
248599530,5rtkit-daemon2484-21rtkit-daemon05:07:462
161272533,43sleep20-21swapper/205:17:472
140722533,44sleep20-21swapper/209:36:332
90762524,41sleep20-21swapper/205:02:482
28792995235,11cyclictest8965-21cut05:02:463
28792995228,9cyclictest25993-21ssh07:28:593
28792995227,8cyclictest24755-21ssh06:42:013
28792995224,12cyclictest0-21swapper/309:28:483
28792995223,12cyclictest0-21swapper/309:32:423
2879299522,45cyclictest0-21swapper/306:17:543
2879299522,44cyclictest11767-21munin-node07:57:543
2879299522,42cyclictest31702-21ssh06:51:393
28792995216,14cyclictest0-21swapper/305:48:473
2879199525,14cyclictest0-21swapper/207:42:522
28791995234,12cyclictest13272-21chrt07:57:552
2879099526,39cyclictest0-21swapper/106:07:511
2879099523,43cyclictest0-21swapper/107:27:481
28790995227,10cyclictest0-21swapper/106:22:591
28790995225,12cyclictest0-21swapper/109:18:071
28790995223,13cyclictest0-21swapper/104:58:331
2878799524,12cyclictest0-21swapper/010:02:150
248599520,5rtkit-daemon2484-21rtkit-daemon09:47:282
248599520,5rtkit-daemon2484-21rtkit-daemon09:15:292
248599520,5rtkit-daemon2484-21rtkit-daemon06:27:282
248599520,5rtkit-daemon2484-21rtkit-daemon05:47:522
2879299519,25cyclictest0-21swapper/309:55:373
2879299519,25cyclictest0-21swapper/309:55:373
2879299515,39cyclictest0-21swapper/307:08:113
2879299515,39cyclictest0-21swapper/307:08:113
28792995133,11cyclictest14035-21head08:47:423
28792995122,8cyclictest5786-21ssh09:22:493
28792995120,10cyclictest1726-21fschecks_time09:17:403
2879199514,27cyclictest0-21swapper/209:57:472
2879199514,27cyclictest0-21swapper/204:47:452
28791995135,10cyclictest26123-21grep08:17:442
2879199513,27cyclictest0-21swapper/207:02:492
28791995123,14cyclictest0-21swapper/207:12:342
28791995121,12cyclictest0-21swapper/209:07:272
28791995121,12cyclictest0-21swapper/209:07:272
28791995117,14cyclictest0-21swapper/206:54:472
2879099514,29cyclictest0-21swapper/105:37:301
2879099514,28cyclictest0-21swapper/106:12:481
2879099513,42cyclictest0-21swapper/105:20:471
248599510,5rtkit-daemon2484-21rtkit-daemon08:43:082
248599510,5rtkit-daemon2484-21rtkit-daemon06:12:292
248599510,4rtkit-daemon2484-21rtkit-daemon04:52:542
248599510,4rtkit-daemon2484-21rtkit-daemon04:37:473
178302512,43sleep30-21swapper/305:20:513
2879299504,38cyclictest0-21swapper/309:06:043
2879299503,40cyclictest0-21swapper/309:37:543
2879299503,29cyclictest0-21swapper/308:02:463
2879299503,27cyclictest0-21swapper/306:44:153
28792995023,11cyclictest0-21swapper/307:03:043
28792995022,10cyclictest1419-21runrttasks07:13:263
28792995018,13cyclictest0-21swapper/305:42:303
2879199504,25cyclictest0-21swapper/205:02:012
28791995034,11cyclictest31730-21cpuspeed_turbos04:42:452
2879199503,28cyclictest0-21swapper/205:12:512
28791995018,10cyclictest16306-21ssh08:02:502
28791995016,15cyclictest0-21swapper/205:52:432
2879099505,27cyclictest0-21swapper/105:57:381
2879099504,40cyclictest0-21swapper/104:52:441
28790995034,10cyclictest32224-21head04:42:511
2879099503,41cyclictest0-21swapper/105:48:031
2879099503,41cyclictest0-21swapper/105:22:591
28790995024,11cyclictest0-21swapper/105:42:431
28790995022,8cyclictest2564-21cat05:57:311
28787995025,12cyclictest0-21swapper/004:48:060
276722502,41sleep330286-21kworker/3:008:20:023
248599500,5rtkit-daemon2484-21rtkit-daemon08:27:382
248599500,5rtkit-daemon2484-21rtkit-daemon08:27:382
248599500,5rtkit-daemon2484-21rtkit-daemon06:48:572
68832492,9sleep20-21swapper/206:03:472
2879299494,41cyclictest0-21swapper/310:06:443
2879299494,37cyclictest0-21swapper/306:13:003
28792994934,9cyclictest17330-21cpuspeed_turbos06:27:443
28792994934,9cyclictest17330-21cpuspeed_turbos06:27:443
28792994933,10cyclictest2703-21grep04:47:573
2879299493,42cyclictest0-21swapper/305:47:153
28792994923,10cyclictest6359-21telnet04:57:433
2879199493,28cyclictest0-21swapper/204:38:012
2879199493,26cyclictest0-21swapper/206:12:412
2879099492,41cyclictest0-21swapper/106:37:121
93422483,38sleep00-21swapper/006:09:450
35592483,39sleep10-21swapper/104:51:301
28792994822,12cyclictest14567-21munin-node06:23:053
28792994820,12cyclictest0-21swapper/306:12:293
28792994816,13cyclictest0-21swapper/309:00:553
2879199483,25cyclictest0-21swapper/205:27:532
28791994817,14cyclictest0-21swapper/206:17:392
2878799485,38cyclictest0-21swapper/005:10:370
2878799485,38cyclictest0-21swapper/005:10:370
28787994833,10cyclictest0-21swapper/005:45:150
28787994828,9cyclictest0-21swapper/004:37:500
141742484,9sleep10-21swapper/106:20:311
28792994719,9cyclictest13420-21munin-node05:12:593
2879099474,38cyclictest0-21swapper/105:16:271
2879099473,27cyclictest0-21swapper/106:27:481
2879099473,27cyclictest0-21swapper/106:27:481
2878799474,37cyclictest0-21swapper/005:17:040
225882473,10sleep30-21swapper/305:30:263
2879299464,36cyclictest0-21swapper/305:32:423
2879299464,35cyclictest0-21swapper/304:52:443
28792994623,8cyclictest1287-21snmpd06:07:083
28790994619,12cyclictest0-21swapper/105:32:311
28787994620,13cyclictest0-21swapper/005:02:430
2878799455,34cyclictest0-21swapper/006:02:560
2878799454,36cyclictest0-21swapper/006:32:450
2878799454,36cyclictest0-21swapper/005:52:590
2878799454,34cyclictest0-21swapper/004:42:430
28787994531,10cyclictest0-21swapper/006:28:270
28787994531,10cyclictest0-21swapper/006:28:270
2878799453,35cyclictest0-21swapper/005:37:440
2878799445,35cyclictest0-21swapper/004:53:010
2879199434,30cyclictest0-21swapper/204:37:342
2878799435,33cyclictest0-21swapper/004:57:490
121642432,35sleep30-21swapper/305:07:573
121642432,35sleep30-21swapper/305:07:573
2879099403,32cyclictest0-21swapper/104:37:341
2879299358,12cyclictest0-21swapper/304:37:333
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional