You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-18 - 01:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Thu Sep 17, 2026 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25061993332,1cyclictest10083-21lspci10:29:290
25061992718,8cyclictest11986-21/usr/sbin/munin12:27:300
2506199270,23cyclictest31033-21ssh09:20:440
25061992624,1cyclictest62550irq/127-eno111:55:000
25061992622,4cyclictest18403-21diskmemload11:48:270
25061992621,5cyclictest0-21swapper/009:57:310
2506199260,21cyclictest7626-21apt-get07:42:310
2506199260,1cyclictest16762-21perf08:00:000
2507199253,21cyclictest8518-21ssh11:25:392
2507199250,24cyclictest0-21swapper/211:47:312
25067992522,0cyclictest18403-21diskmemload12:36:101
2506799250,24cyclictest18403-21diskmemload09:15:531
2506799250,24cyclictest14079-21apt-get09:02:301
25061992520,4cyclictest0-21swapper/012:12:320
2506199250,24cyclictest0-21swapper/009:36:020
2506199250,24cyclictest0-21swapper/009:16:540
25075992423,1cyclictest0-21swapper/312:17:313
25075992421,2cyclictest5378-21ssh10:26:063
25075992420,3cyclictest29868-21turbostat.cron07:20:003
2507599240,24cyclictest0-21swapper/310:22:413
2507599240,24cyclictest0-21swapper/309:58:433
2507599240,23cyclictest22472-21perf10:40:013
2507599240,23cyclictest18403-21diskmemload10:58:313
2507599240,23cyclictest0-21swapper/310:34:433
25071992423,1cyclictest31327-21apt-get11:17:322
2507199240,24cyclictest18403-21diskmemload12:32:182
2507199240,23cyclictest26870-21perf12:40:002
2507199240,23cyclictest0-21swapper/210:02:332
25067992424,0cyclictest0-21swapper/111:28:101
25067992422,2cyclictest21645-21ssh12:06:361
25067992422,1cyclictest32253-21apt-get09:22:301
25067992422,1cyclictest0-21swapper/109:55:521
2506799240,23cyclictest18403-21diskmemload11:49:171
2506799240,23cyclictest11304-21ssh10:58:571
2506799240,23cyclictest0-21swapper/109:32:311
2506799240,23cyclictest0-21swapper/107:52:301
2506799240,1cyclictest30643-21ssh10:19:031
2506799240,1cyclictest18235-21cat11:05:011
2506799240,1cyclictest0-21swapper/111:13:421
25061992420,3cyclictest30092-21sh09:49:200
2506199240,23cyclictest18403-21diskmemload10:45:340
2506199240,23cyclictest0-21swapper/009:11:310
2506199240,0cyclictest18403-21diskmemload09:40:470
2506199240,0cyclictest0-21swapper/012:31:540
2506199240,0cyclictest0-21swapper/010:30:100
2506199240,0cyclictest0-21swapper/009:33:140
25075992323,0cyclictest18403-21diskmemload11:23:543
25075992323,0cyclictest0-21swapper/309:16:083
25075992321,1cyclictest27146-21ssh12:11:233
25075992320,2cyclictest9590-21ssh09:30:583
25075992320,2cyclictest21562-21latency_hist08:10:003
25075992320,2cyclictest16977-21switchtime11:32:403
25075992320,2cyclictest11705-21sh12:27:083
2507599230,2cyclictest5791-21sh11:52:303
2507599230,23cyclictest18403-21diskmemload12:33:153
2507599230,23cyclictest18403-21diskmemload10:05:273
2507599230,23cyclictest0-21swapper/311:55:443
2507599230,23cyclictest0-21swapper/311:40:323
2507599230,23cyclictest0-21swapper/310:51:373
2507599230,22cyclictest5591-21ssh12:21:073
2507599230,22cyclictest31022-21sh09:20:433
2507599230,22cyclictest18403-21diskmemload09:27:193
2507599230,0cyclictest18403-21diskmemload10:44:363
2507599230,0cyclictest18403-21diskmemload09:52:093
2507599230,0cyclictest0-21swapper/311:09:273
2507599230,0cyclictest0-21swapper/311:00:333
2507599230,0cyclictest0-21swapper/310:11:423
25071992323,0cyclictest0-21swapper/211:33:092
25071992323,0cyclictest0-21swapper/210:38:272
25071992322,1cyclictest18403-21diskmemload12:11:362
25071992322,1cyclictest18403-21diskmemload10:58:352
25071992321,1cyclictest22378-21apt-get12:07:312
25071992321,1cyclictest13621-21tr09:00:012
25071992321,1cyclictest13621-21tr09:00:012
25071992321,1cyclictest10874-21ssh11:56:562
25071992320,2cyclictest1009-21basename10:50:002
25071992320,0cyclictest0-21swapper/210:05:302
2507199230,2cyclictest28314-21ssh11:13:452
2507199230,2cyclictest21064-21ssh10:10:482
2507199230,23cyclictest0-21swapper/212:29:352
2507199230,23cyclictest0-21swapper/212:20:562
2507199230,23cyclictest0-21swapper/211:42:202
2507199230,23cyclictest0-21swapper/209:51:102
2507199230,22cyclictest21075-21ssh09:41:472
2507199230,22cyclictest20266-21ssh09:12:002
2507199230,22cyclictest15780-21rm10:33:302
2507199230,22cyclictest10447-21sh09:32:182
2507199230,0cyclictest18403-21diskmemload10:41:592
2507199230,0cyclictest0-21swapper/209:58:462
25067992323,0cyclictest0-21swapper/111:53:281
25067992323,0cyclictest0-21swapper/110:47:031
25067992323,0cyclictest0-21swapper/110:37:371
25067992323,0cyclictest0-21swapper/110:26:131
25067992321,1cyclictest31527-21ssh09:51:001
25067992321,1cyclictest23294-21missed_timers10:12:361
25067992321,1cyclictest0-21swapper/112:11:591
25067992321,1cyclictest0-21swapper/109:43:141
25067992321,1cyclictest0-21swapper/109:10:161
2506799230,2cyclictest14987-21ssh10:05:021
2506799230,23cyclictest0-21swapper/111:15:221
2506799230,22cyclictest1881-21sh10:50:491
2506799230,22cyclictest15550-21munin-plugin-st12:30:001
2506799230,22cyclictest0-21swapper/112:01:041
2506799230,22cyclictest0-21swapper/111:20:201
2506799230,1cyclictest0-21swapper/112:31:421
2506799230,0cyclictest0-21swapper/111:33:551
2506799230,0cyclictest0-21swapper/109:46:401
25061992323,0cyclictest18403-21diskmemload11:15:300
25061992321,1cyclictest17670-21idleruntime-cro11:05:000
25061992321,1cyclictest0-21swapper/010:43:310
25061992320,3cyclictest0-21swapper/011:42:450
25061992320,3cyclictest0-21swapper/010:22:290
25061992320,2cyclictest0-21swapper/012:05:190
2506199230,23cyclictest0-21swapper/011:39:240
2506199230,23cyclictest0-21swapper/011:27:080
2506199230,23cyclictest0-21swapper/010:36:100
2506199230,22cyclictest9546-21ssh11:55:060
2506199230,22cyclictest19506-21ssh10:08:590
2506199230,22cyclictest16343-21ssh12:01:540
2506199230,22cyclictest11121-21ssh10:58:430
2506199230,0cyclictest31332-21ssh09:50:450
2506199230,0cyclictest18403-21diskmemload11:14:340
2506199230,0cyclictest0-21swapper/012:15:450
2506199230,0cyclictest0-21swapper/011:21:520
2506199230,0cyclictest0-21swapper/010:00:400
25075992222,0cyclictest0-21swapper/311:26:263
25075992222,0cyclictest0-21swapper/310:18:463
25075992222,0cyclictest0-21swapper/309:40:283
25075992222,0cyclictest0-21swapper/309:12:513
25075992221,1cyclictest21413-21ssh12:06:073
25075992219,2cyclictest24770-21ssh11:11:223
2507599220,21cyclictest28276-21ssh10:45:133
2507599220,21cyclictest10017-21apt-get07:47:293
2507599220,21cyclictest0-21swapper/312:00:253
2507599220,1cyclictest31017-21ssh11:17:103
2507599220,1cyclictest23839-21ssh11:38:593
2507599220,1cyclictest16762-21perf08:00:003
2507599220,1cyclictest0-21swapper/312:35:293
2507599220,0cyclictest31050-21ssh11:45:223
25071992222,0cyclictest0-21swapper/210:52:132
25071992222,0cyclictest0-21swapper/207:37:312
25071992220,1cyclictest26024-21sh10:15:052
25071992220,1cyclictest16784-21awk08:00:002
25071992220,1cyclictest0-21swapper/211:35:282
2507199220,22cyclictest18403-21diskmemload10:22:162
2507199220,22cyclictest0-21swapper/210:25:322
2507199220,21cyclictest3545-21ssh09:25:012
2507199220,21cyclictest19882-21ssh11:07:242
2507199220,21cyclictest18403-21diskmemload12:15:512
2507199220,21cyclictest18403-21diskmemload11:50:102
2507199220,21cyclictest18403-21diskmemload09:20:012
2507199220,21cyclictest15614-21ssh12:00:452
2507199220,21cyclictest12766-21sh11:00:402
2507199220,0cyclictest26814-21apt-get09:17:312
2507199220,0cyclictest0-21swapper/209:45:282
25067992222,0cyclictest18403-21diskmemload12:20:221
25067992222,0cyclictest0-21swapper/111:41:191
25067992222,0cyclictest0-21swapper/109:35:581
25067992220,1cyclictest17224-21apt-get08:02:301
2506799220,22cyclictest16469-21ssh11:03:171
2506799220,21cyclictest0-21swapper/111:55:371
2506799220,21cyclictest0-21swapper/111:36:091
2506799220,21cyclictest0-21swapper/110:40:191
2506799220,21cyclictest0-21swapper/110:01:341
2506799220,1cyclictest0-21swapper/110:22:331
2506799220,1cyclictest0-21swapper/109:26:161
25061992220,2cyclictest20523-21ssh10:10:060
25061992220,1cyclictest11664-21apt-get08:57:320
25061992220,1cyclictest11664-21apt-get08:57:320
25061992220,1cyclictest0-21swapper/011:31:180
2506199220,22cyclictest18403-21diskmemload10:18:000
2506199220,21cyclictest18422-21ssh11:05:090
2506199220,21cyclictest0-21swapper/008:32:300
2506199220,21cyclictest0-21swapper/007:22:300
2506199220,1cyclictest2709-21apt-get07:32:300
2506199220,1cyclictest1720-21taskset10:50:370
2506199220,1cyclictest0-21swapper/012:37:010
25075992121,0cyclictest0-21swapper/307:32:303
25075992120,1cyclictest18403-21diskmemload09:35:113
25075992120,0cyclictest0-21swapper/308:57:323
25075992120,0cyclictest0-21swapper/308:57:323
2507599210,21cyclictest0-21swapper/310:00:233
2507599210,21cyclictest0-21swapper/309:45:023
2507599210,20cyclictest2060-21apt-get08:37:303
2507599210,20cyclictest12413-21apt-get07:52:313
2507599210,20cyclictest0-21swapper/308:52:303
25071992120,0cyclictest4481-21apt-get08:42:322
25071992119,1cyclictest2709-21sh11:20:152
2507199210,20cyclictest2270-21awk07:30:012
2507199210,20cyclictest21580-21munin-plugin-st08:10:012
2507199210,0cyclictest0-21swapper/209:35:362
2507199210,0cyclictest0-21swapper/207:17:302
2506799210,21cyclictest0-21swapper/112:15:191
2506799210,21cyclictest0-21swapper/110:30:071
2506799210,20cyclictest0-21swapper/107:47:281
2506799210,1cyclictest0-21swapper/108:27:321
25061992121,0cyclictest0-21swapper/008:02:300
25061992120,1cyclictest7589-21ssh09:28:230
25061992120,1cyclictest0-21swapper/008:12:300
25075992018,1cyclictest19186-21awk08:05:003
2507599200,0cyclictest0-21swapper/308:27:333
25071992020,0cyclictest0-21swapper/209:07:312
25071992019,0cyclictest0-21swapper/207:12:302
25071992017,2cyclictest31616-21dump-pmu-power08:30:002
2507199200,19cyclictest30328-21apt-get07:22:302
25067992020,0cyclictest0-21swapper/108:57:321
25067992020,0cyclictest0-21swapper/108:57:321
25067992020,0cyclictest0-21swapper/108:12:301
25067992020,0cyclictest0-21swapper/107:27:311
25067992020,0cyclictest0-21swapper/107:17:301
25061992020,0cyclictest0-21swapper/007:27:310
25061992020,0cyclictest0-21swapper/007:17:300
2506199200,0cyclictest0-21swapper/008:47:320
2506199200,0cyclictest0-21swapper/007:47:280
25075991919,0cyclictest6877-21apt-get08:47:323
25075991919,0cyclictest0-21swapper/308:17:323
2507599190,0cyclictest0-21swapper/308:42:323
25071991918,1cyclictest22016-21apt-get08:12:302
25067991918,0cyclictest0-21swapper/108:45:001
25067991917,1cyclictest32290-21perf07:25:011
25061991919,0cyclictest0-21swapper/008:22:310
25061991917,1cyclictest6433-21idleruntime-cro08:45:000
25075991818,0cyclictest0-21swapper/308:22:313
25071991818,0cyclictest32077-21apt-get08:32:302
25071991818,0cyclictest0-21swapper/207:42:312
25071991816,1cyclictest11205-21sh08:55:002
25067991818,0cyclictest0-21swapper/108:47:321
25067991818,0cyclictest0-21swapper/108:07:281
2506799180,0cyclictest0-21swapper/108:52:301
25061991818,0cyclictest0-21swapper/008:17:320
25061991818,0cyclictest0-21swapper/007:12:300
25075991717,0cyclictest0-21swapper/309:07:313
25075991717,0cyclictest0-21swapper/308:32:303
2507599170,0cyclictest0-21swapper/307:37:303
25071991717,0cyclictest0-21swapper/209:02:302
25071991717,0cyclictest0-21swapper/207:52:312
25071991717,0cyclictest0-21swapper/207:47:292
25071991717,0cyclictest0-21swapper/207:32:302
25071991716,0cyclictest26838-21apt-get08:22:312
2507199170,0cyclictest0-21swapper/208:02:302
25067991717,0cyclictest0-21swapper/108:32:301
25061991717,0cyclictest0-21swapper/008:37:300
25075991616,0cyclictest0-21swapper/309:02:303
25075991616,0cyclictest0-21swapper/308:12:303
25075991616,0cyclictest0-21swapper/307:12:303
25075991614,1cyclictest0-21swapper/307:27:373
25071991616,0cyclictest0-21swapper/208:37:302
2506799165,10cyclictest25461-21apt-get07:12:301
25067991616,0cyclictest0-21swapper/108:22:311
2506799160,15cyclictest0-21swapper/108:37:301
25071991515,0cyclictest0-21swapper/208:47:322
25067991513,1cyclictest7140-21cron07:40:001
2506799150,5cyclictest0-21swapper/107:32:301
2506799150,1cyclictest26343-21cron08:20:011
2506799150,1cyclictest0-21swapper/109:07:391
2506199152,12cyclictest611-21irqbalance09:09:190
25061991515,0cyclictest0-21swapper/008:07:290
25061991514,1cyclictest0-21swapper/008:26:490
2506199150,1cyclictest13054-21hddtemp_smartct07:52:330
2506199150,15cyclictest0-21swapper/009:03:250
2507599140,14cyclictest0-21swapper/307:41:113
2507599140,0cyclictest0-21swapper/307:20:303
2507199140,0cyclictest0-21swapper/208:15:022
25067991414,0cyclictest0-21swapper/107:55:091
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional