You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-25 - 20:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Tue Aug 25, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2585129365,9sleep00-21swapper/007:05:090
2584429263,9sleep30-21swapper/307:05:043
2776728874,10sleep20-21swapper/207:07:162
2764628669,13sleep10-21swapper/107:05:431
172162600,0sleep30-21swapper/312:20:283
71292470,0sleep10-21swapper/110:18:561
28074992727,0cyclictest0-21swapper/310:22:443
2806899270,1cyclictest0-21swapper/110:52:231
2807499252,22cyclictest0-21swapper/309:13:533
2806899250,4cyclictest0-21swapper/110:35:011
28074992423,1cyclictest0-21swapper/309:17:573
2806899240,0cyclictest0-21swapper/109:36:101
2807499220,19cyclictest0-21swapper/310:15:143
2807199220,19cyclictest0-21swapper/209:20:512
2806899221,18cyclictest0-21swapper/112:20:221
2806899220,1cyclictest0-21swapper/110:01:171
28074992117,3cyclictest12968-21kworker/3:009:22:423
2807499210,18cyclictest0-21swapper/311:34:193
2806499210,2cyclictest0-21swapper/012:02:210
2806899203,11cyclictest38150irq/131-snd_hda07:15:141
2806899202,9cyclictest0-21swapper/108:05:141
2806899200,3cyclictest0-21swapper/111:36:301
28074991918,1cyclictest0-21swapper/309:45:133
2807199190,18cyclictest0-21swapper/209:51:122
28068991914,5cyclictest38150irq/131-snd_hda11:58:491
28068991914,2cyclictest0-21swapper/108:20:111
2806899190,1cyclictest0-21swapper/112:39:561
2806899190,0cyclictest0-21swapper/112:31:181
92882180,0sleep10-21swapper/109:21:261
2807499182,15cyclictest0-21swapper/312:05:153
28071991817,1cyclictest0-21swapper/209:58:122
2807199180,17cyclictest0-21swapper/212:10:222
28068991817,1cyclictest0-21swapper/111:10:241
28068991815,2cyclictest0-21swapper/107:40:201
2807499170,1cyclictest0-21swapper/311:48:153
2807499170,16cyclictest0-21swapper/309:44:143
28071991712,2cyclictest0-21swapper/211:47:182
2807199170,1cyclictest0-21swapper/209:15:132
2806899175,8cyclictest0-21swapper/111:40:141
2806899171,1cyclictest7584-21munin-run07:35:001
2806499177,7cyclictest477-21rs:main0
2806499176,7cyclictest13-21ksoftirqd/011:53:380
2806499174,1cyclictest0-21swapper/011:55:130
28064991716,1cyclictest0-21swapper/012:17:160
28064991712,3cyclictest9740-21ls10:00:260
2806499170,16cyclictest0-21swapper/010:52:350
28074991615,1cyclictest0-21swapper/311:22:003
28074991614,1cyclictest0-21swapper/312:16:023
28074991611,3cyclictest484-21NetworkManager11:53:433
2807199162,12cyclictest49-21kcompactd010:34:012
28068991616,0cyclictest0-21swapper/110:22:441
28068991615,1cyclictest0-21swapper/110:28:591
28068991615,1cyclictest0-21swapper/107:10:201
28068991612,4cyclictest25-21ksoftirqd/108:35:121
2806899160,16cyclictest0-21swapper/111:30:341
28064991612,3cyclictest20869-21diskmemload11:04:150
2807199150,1cyclictest0-21swapper/210:51:162
2806899150,14cyclictest0-21swapper/112:26:371
2806499154,6cyclictest17311-21munin-run07:55:000
2806499151,3cyclictest0-21swapper/007:41:550
28064991512,2cyclictest10357-21sh11:00:000
2806499150,14cyclictest0-21swapper/011:09:300
2807499146,4cyclictest225-21systemd-journal07:55:003
28074991410,2cyclictest8471-21sed08:40:253
28074991410,2cyclictest7455-21wc11:55:253
28074991410,2cyclictest13318-21ssh11:40:153
2807499140,2cyclictest0-21swapper/310:56:433
2807199148,5cyclictest1026-21ssh09:35:422
2807199143,10cyclictest20869-21diskmemload09:10:492
2807199142,3cyclictest20869-21diskmemload11:15:052
28071991411,2cyclictest10837-21grep10:20:242
2807199140,13cyclictest1421-21sed07:20:172
2806899140,13cyclictest0-21swapper/111:20:141
2806499149,3cyclictest32299-21wc09:35:190
28064991410,3cyclictest13767-21ssh11:20:270
2806499140,12cyclictest20195-21grep10:45:250
2806499140,0cyclictest0-21swapper/008:10:130
2807499139,3cyclictest3516-21grep07:25:153
2807499139,3cyclictest19386-21sed07:55:263
2807499139,2cyclictest18796-21grep10:45:143
2807499139,2cyclictest11328-21cron08:50:003
2807499138,3cyclictest29055-21cut07:10:243
28074991313,0cyclictest0-21swapper/309:25:263
28074991310,2cyclictest8890-21head12:35:173
28074991310,2cyclictest3501-21grep10:35:263
28074991310,2cyclictest27354-21basename08:15:143
28074991310,2cyclictest17143-21sed09:00:183
28074991310,2cyclictest17143-21sed09:00:183
28074991310,2cyclictest15621-21grep08:55:263
28074991310,2cyclictest14917-21cron07:50:003
2807199139,3cyclictest5123-21tr11:35:142
2807199139,3cyclictest20274-21tr08:00:122
2807199137,3cyclictest32-21ksoftirqd/210:10:482
28071991310,2cyclictest9240-21ls07:35:212
28071991310,2cyclictest30536-21grep11:50:172
28071991310,2cyclictest26491-21cat08:10:252
28071991310,2cyclictest16075-21cron09:00:012
2806899138,4cyclictest788-21egrep07:20:141
28068991312,1cyclictest0-21swapper/109:17:181
28068991310,2cyclictest736-21wc09:55:191
28068991310,2cyclictest22246-21idleruntime-cro09:30:001
28068991310,2cyclictest19737-21cat11:05:151
28068991310,2cyclictest15033-21cat07:50:001
28068991310,2cyclictest1270-21grep08:25:261
2806899130,13cyclictest0-21swapper/111:45:151
2806899130,12cyclictest25737-21ssh11:28:071
2806499139,3cyclictest22716-21tr11:45:200
2806499135,4cyclictest80062sleep007:35:030
2806499132,3cyclictest4312-21perf11:35:010
28064991310,2cyclictest5968-21wc08:35:250
28064991310,2cyclictest3737-21grep07:25:180
28064991310,2cyclictest364-21sed08:25:170
28064991310,2cyclictest29929-21tr08:20:130
28064991310,2cyclictest29539-21grep07:10:280
28064991310,2cyclictest25938-21tail10:10:200
28064991310,2cyclictest23875-21basename09:50:140
28064991310,2cyclictest20917-21sed08:00:160
28064991310,2cyclictest10833-21tail10:40:180
2806499130,5cyclictest0-21swapper/010:30:480
2806499130,1cyclictest0-21swapper/012:34:320
2807499129,2cyclictest32428-21tr12:30:163
2807499129,2cyclictest29505-21grep11:10:263
2807499129,2cyclictest26922-21grep10:50:163
2807499129,2cyclictest21133-21grep11:25:173
2807499129,2cyclictest17765-21dump-pmu-power10:45:003
2807499129,2cyclictest12149-21grep08:50:153
2807499128,2cyclictest6141-21grep08:35:253
2807499128,2cyclictest22842-21grep09:30:153
2807499128,2cyclictest17808-21tr10:05:233
2807499124,2cyclictest32045-21apt-get09:55:133
28074991211,1cyclictest17282sleep312:11:283
2807499120,11cyclictest11020-21tr11:00:153
2807199129,2cyclictest9935-21grep08:45:172
2807199129,2cyclictest938-21cut10:15:152
2807199129,2cyclictest9300-21ntpq12:35:202
2807199129,2cyclictest8660-21rm10:00:162
2807199129,2cyclictest7656-21grep11:55:262
2807199129,2cyclictest7136-21tr08:40:142
2807199129,2cyclictest5690-21tr08:35:212
2807199129,2cyclictest31900-21grep07:15:222
2807199129,2cyclictest29706-21tail11:30:162
2807199129,2cyclictest2815-21grep10:55:152
2807199129,2cyclictest24072-21cat09:30:222
2807199129,2cyclictest2385-21sed10:35:162
2807199129,2cyclictest22994-21idleruntime-cro12:25:002
2807199129,2cyclictest21998-21cron12:05:002
2807199129,2cyclictest21056-21grep11:25:172
2807199129,2cyclictest19012-21expr09:05:152
2807199129,2cyclictest18975-21cut10:45:162
2807199129,2cyclictest18072-21grep09:00:272
2807199129,2cyclictest18072-21grep09:00:272
2807199129,2cyclictest14010-21grep11:20:272
2807199129,2cyclictest13892-21grep11:40:172
2807199129,2cyclictest11199-21sed07:40:182
2807199126,4cyclictest13926-21cut07:45:212
2807199120,4cyclictest0-21swapper/212:06:172
2807199120,12cyclictest0-21swapper/209:43:222
2807199120,10cyclictest0-21swapper/212:16:222
2806899129,2cyclictest61332sleep111:15:471
2806899129,2cyclictest304-21grep12:10:261
2806899129,2cyclictest22820-21tr12:05:161
2806899129,2cyclictest19514-21head09:05:181
2806899129,2cyclictest19372-21grep10:45:181
2806899129,2cyclictest18073-21grep09:00:271
2806899129,2cyclictest18073-21grep09:00:271
2806899129,2cyclictest11057-21grep11:00:151
2806899129,2cyclictest10890-21grep08:45:271
2806899129,2cyclictest10345-21cron11:00:001
2806899128,3cyclictest4097-21apache212:03:311
2806899125,6cyclictest17777-21turbostat.cron10:45:001
2806899123,7cyclictest21581-21apt-get09:10:131
2806899122,6cyclictest31738-21grep11:50:271
28068991211,1cyclictest0-21swapper/109:42:221
2806899120,12cyclictest0-21swapper/112:18:461
2806499129,2cyclictest9912-21sed12:35:260
2806499129,2cyclictest6229-21sed07:30:160
2806499129,2cyclictest30751-21sh09:15:160
2806499129,2cyclictest28037-21grep08:15:170
2806499129,2cyclictest24661-21latency_hist08:10:000
2806499129,2cyclictest20470-21cut11:25:140
2806499129,2cyclictest19392-21grep07:55:260
2806499129,2cyclictest16459-21tr12:20:210
2806499129,2cyclictest14441-21sed07:45:270
2806499128,3cyclictest31024-21grep12:10:140
2806499128,3cyclictest22370-21ssh09:10:170
2806499124,2cyclictest7536-21munin-run10:00:000
28064991210,1cyclictest37152sleep011:15:140
2806499120,11cyclictest23567-21cut12:25:140
2807499118,2cyclictest894-21ssh09:35:363
2807499118,2cyclictest4675-21ssh11:15:183
2807499118,2cyclictest4625-21ssh11:35:053
2807499118,2cyclictest3287-21wc08:30:243
2807499118,2cyclictest31415-21tail07:15:173
2807499118,2cyclictest30414-21tail08:20:153
2807499118,2cyclictest27847-21ssh10:30:373
2807499118,2cyclictest25662-21ssh12:25:323
2807499118,2cyclictest24698-21sh10:10:123
2807499118,2cyclictest24003-21tr09:50:163
2807499118,2cyclictest21975-21ssh11:06:003
2807499118,2cyclictest206212sleep309:07:103
2807499118,2cyclictest19230-21grep10:25:283
2807499118,2cyclictest16241-21taskset12:00:283
2807499118,2cyclictest10259-21ssh10:00:363
2807499114,3cyclictest0-21swapper/307:35:033
2807499113,4cyclictest225-21systemd-journal07:44:593
2807199119,1cyclictest21157-21grep11:05:262
2807199118,2cyclictest5748-21tr07:30:142
2807199118,2cyclictest5194-21idleruntime-cro07:30:002
2807199118,2cyclictest3935-21taskset08:32:362
2807199118,2cyclictest2858-21ssh12:31:222
2807199118,2cyclictest27848-21grep11:10:152
2807199118,2cyclictest27550-21tr08:15:142
2807199118,2cyclictest25948-21sh12:25:472
2807199118,2cyclictest18945-21ssh10:25:252
2807199118,2cyclictest16448-21sh09:25:292
2807199118,2cyclictest15827-21expr09:45:142
2807199118,2cyclictest13498-21sh10:41:382
2807199118,2cyclictest1326-21sed08:25:262
2807199118,2cyclictest10905-21ssh11:00:132
2807199110,10cyclictest16665-21grep10:05:152
2806899119,1cyclictest3695-21sed08:30:281
2806899118,2cyclictest27799-21ssh10:30:341
2806899118,2cyclictest25397-21forks10:10:171
2806899118,2cyclictest15723-21grep09:45:141
2806899115,2cyclictest16349-21/usr/sbin/munin10:05:201
2806499118,2cyclictest646-21apt-get07:20:130
2806499118,2cyclictest3569-21ssh10:35:270
2806499118,2cyclictest28426-21grep11:10:170
2806499118,2cyclictest23479-21sh12:05:200
2806499118,2cyclictest20391-21ssh10:26:170
2806499118,2cyclictest19020-21grep09:05:160
2806499118,2cyclictest18873-21ssh09:46:360
2806499118,2cyclictest16174-21ssh10:05:050
2806499118,2cyclictest15547-21wc08:55:260
2806499118,2cyclictest14938-21ssh09:45:000
2806499118,2cyclictest13284-21tr11:40:150
2806499118,2cyclictest12606-21sed08:50:190
2806499117,3cyclictest7631-21cut08:40:160
2806499111,9cyclictest0-21swapper/007:15:140
2806499110,4cyclictest0-21swapper/010:15:140
2806499110,4cyclictest0-21swapper/009:30:160
2807499108,1cyclictest946-21ps08:25:223
2807499108,1cyclictest73552sleep307:32:253
2807499108,1cyclictest20981-21idleruntime08:00:163
2807499107,2cyclictest26255-21cat08:10:223
2807499105,3cyclictest22550-21apache_volume08:05:133
2807199108,1cyclictest22896-21expr08:05:152
2807199108,1cyclictest17212-21seq07:54:022
2807199108,1cyclictest12911-21gzip08:50:222
2807199107,2cyclictest31154-21cat08:20:242
2807199100,6cyclictest0-21swapper/207:55:212
2806899108,1cyclictest8539-21grep07:35:161
2806899108,1cyclictest136042sleep108:53:491
2806899107,2cyclictest7239-21expr08:40:141
2806899107,2cyclictest3224-21tr07:25:141
2806899107,2cyclictest27544-21tr08:15:141
2806899107,2cyclictest26844-21rm09:32:041
2806899107,2cyclictest24923-21cat09:50:201
2806499108,1cyclictest3588-21cat08:30:280
2806499107,2cyclictest6776-21grep11:35:270
2806499107,2cyclictest6409-21expr09:20:150
2806499107,2cyclictest18172-21taskset09:01:130
2806499107,2cyclictest18172-21taskset09:01:130
2806499107,2cyclictest11334-21sh08:50:000
2806499101,2cyclictest0-21swapper/010:20:110
280749994,2cyclictest225-21systemd-journal07:25:003
280719990,9cyclictest29467-21ntpq07:10:292
280689998,1cyclictest18931-21ntpq07:55:211
280689997,1cyclictest15171-21cat07:50:111
280689997,1cyclictest15038-21cat08:55:191
280689996,2cyclictest24888-21seq08:10:091
280689992,3cyclictest225-21systemd-journal08:03:341
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional