You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-31 - 18:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sun May 31, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2955929280,8sleep00-21swapper/007:06:040
2965429165,8sleep30-21swapper/307:07:193
2985728777,6sleep20-21swapper/207:09:552
2953528568,13sleep10-21swapper/107:05:441
130882710,0sleep30-21swapper/309:23:103
2994899280,28cyclictest0-21swapper/211:04:482
2994899280,28cyclictest0-21swapper/209:44:442
2994399280,0cyclictest0-21swapper/112:08:301
2994899250,25cyclictest0-21swapper/211:14:482
2994899250,25cyclictest0-21swapper/209:12:392
2994399253,10cyclictest0-21swapper/111:50:131
60352240,2sleep232-21ksoftirqd/211:34:242
29939992423,1cyclictest0-21swapper/010:02:020
2993999240,1cyclictest0-21swapper/011:36:380
2993999240,1cyclictest0-21swapper/011:22:110
2993999240,1cyclictest0-21swapper/009:58:190
2994899230,21cyclictest0-21swapper/212:10:022
2993999230,1cyclictest0-21swapper/011:49:540
2993999230,0cyclictest0-21swapper/011:02:190
2994899220,22cyclictest0-21swapper/211:41:032
2994899220,22cyclictest0-21swapper/211:17:082
29939992222,0cyclictest0-21swapper/010:28:150
2993999220,1cyclictest0-21swapper/010:58:200
29948992120,1cyclictest0-21swapper/209:53:072
2993999210,2cyclictest0-21swapper/009:11:270
2995399204,8cyclictest0-21swapper/310:00:143
29948992016,1cyclictest32-21ksoftirqd/210:31:222
29948992015,1cyclictest32-21ksoftirqd/210:37:322
2994899200,1cyclictest0-21swapper/210:11:512
2993999200,3cyclictest0-21swapper/011:28:520
2993999200,3cyclictest0-21swapper/009:24:090
2993999200,20cyclictest0-21swapper/010:41:000
2993999200,1cyclictest0-21swapper/010:46:570
2994899191,17cyclictest0-21swapper/209:45:162
2994899190,19cyclictest0-21swapper/211:39:572
2994399191,17cyclictest22737-21diskmemload10:16:511
29939991919,0cyclictest0-21swapper/011:56:360
2993999190,4cyclictest0-21swapper/011:32:590
2993999190,2cyclictest0-21swapper/012:03:130
2993999190,1cyclictest0-21swapper/012:18:180
245382190,0sleep20-21swapper/210:08:462
2994399182,13cyclictest9868-21apache_volume10:00:131
324152170,0sleep10-21swapper/111:10:351
2995399172,10cyclictest4398-21/usr/sbin/munin12:30:233
29948991716,1cyclictest0-21swapper/211:59:462
2994899171,1cyclictest0-21swapper/210:25:282
2994899171,15cyclictest0-21swapper/210:48:542
2994899170,17cyclictest0-21swapper/210:01:402
2994899170,16cyclictest0-21swapper/212:00:112
2994399170,0cyclictest0-21swapper/110:33:101
2993999170,3cyclictest0-21swapper/011:51:160
2993999170,0cyclictest0-21swapper/012:24:110
2993999170,0cyclictest0-21swapper/009:34:120
29953991615,1cyclictest0-21swapper/311:49:523
29953991614,2cyclictest39-21ksoftirqd/311:13:093
29953991613,2cyclictest18903-21kworker/u8:010:44:133
29953991611,3cyclictest28311-21sed08:10:263
2994899160,13cyclictest0-21swapper/210:20:472
2994899160,0cyclictest0-21swapper/209:27:182
2994399160,1cyclictest0-21swapper/112:24:101
2993999161,0cyclictest0-21swapper/010:52:180
2993999160,1cyclictest0-21swapper/009:41:210
2993999160,0cyclictest0-21swapper/009:38:470
2995399156,1cyclictest1142-21runrttasks09:45:593
2995399154,10cyclictest14562-21apache212:10:383
29953991515,0cyclictest0-21swapper/310:25:173
29953991511,3cyclictest1055-21grep08:25:013
29948991513,1cyclictest0-21swapper/210:18:272
2994899150,15cyclictest0-21swapper/212:17:552
2994899150,14cyclictest0-21swapper/209:37:322
29943991515,0cyclictest0-21swapper/111:40:551
29943991510,4cyclictest38150irq/131-snd_hda09:30:571
2994399150,14cyclictest0-21swapper/110:45:201
2994399150,0cyclictest0-21swapper/111:30:141
29939991514,1cyclictest0-21swapper/010:05:040
2993999150,0cyclictest0-21swapper/010:14:290
2995399149,4cyclictest12940-21grep10:20:263
29953991410,2cyclictest10634-21grep09:40:253
2995399140,13cyclictest11800-21wc08:45:183
29948991410,3cyclictest24940-21grep08:05:172
2994899140,14cyclictest0-21swapper/211:49:202
2994899140,14cyclictest0-21swapper/210:50:312
2994899140,14cyclictest0-21swapper/209:18:042
2994899140,13cyclictest0-21swapper/211:08:052
2994899140,13cyclictest0-21swapper/209:30:382
2994899140,0cyclictest141rcu_preempt09:22:282
2994399148,5cyclictest0-21swapper/110:50:131
2994399142,4cyclictest225-21systemd-journal08:25:001
29943991411,2cyclictest17745-21cron09:00:011
2993999146,2cyclictest1070-21munin-run12:10:010
2995399139,3cyclictest30897-21tail07:10:173
2995399139,3cyclictest19719-21grep09:00:273
2995399139,3cyclictest1526-21sed09:35:213
2995399139,2cyclictest2316-21wc08:25:193
2995399139,2cyclictest20178-21grep10:05:273
2995399139,2cyclictest13229-21wc07:40:193
2995399133,7cyclictest22737-21diskmemload12:15:123
29953991310,2cyclictest9251-21grep11:35:273
29953991310,2cyclictest4368-21wc10:15:273
29953991310,2cyclictest4303-21grep10:35:163
29953991310,2cyclictest31968-21df11:30:153
29953991310,2cyclictest2790-21grep07:20:143
29953991310,2cyclictest24477-21gzip11:25:223
2995399130,12cyclictest5167-21tr07:25:153
2994899139,3cyclictest22550-21grep08:00:182
2994899139,2cyclictest15979-21wc11:20:192
29948991310,2cyclictest8970-21grep07:30:262
29948991310,2cyclictest31399-21grep07:10:222
29948991310,2cyclictest27687-21grep12:05:292
29948991310,2cyclictest14269-21wc08:50:202
29948991310,2cyclictest13839-21wc07:40:262
2994899130,13cyclictest0-21swapper/210:58:212
2994399139,3cyclictest4792-21grep08:30:221
2994399139,3cyclictest26571-21cat09:50:191
2994399139,3cyclictest10359-21ssh09:40:251
2994399138,4cyclictest11379-21ssh11:37:021
2994399138,3cyclictest4539-21ssh10:35:171
2994399132,1cyclictest22737-21diskmemload10:23:241
29943991310,2cyclictest9981-21cut07:35:141
29943991310,2cyclictest17984-21cut09:45:171
29943991310,2cyclictest1392-21ssh09:35:191
29943991310,2cyclictest0-21swapper/112:38:431
2994399130,13cyclictest0-21swapper/109:55:121
2993999139,4cyclictest95822sleep010:38:250
2993999136,6cyclictest20165-21munin-run09:05:000
2993999135,7cyclictest18914-21munin-run10:25:000
2993999134,5cyclictest475-21in:imuxsock11:14:590
29939991311,1cyclictest22737-21diskmemload11:15:130
29939991310,2cyclictest26769-21tr09:50:190
29939991310,2cyclictest14914-21grep08:50:260
29939991310,2cyclictest12975-21grep07:40:180
2993999130,0cyclictest0-21swapper/012:30:020
2995399129,2cyclictest9347-21awk10:00:003
2995399129,2cyclictest887-21cut07:15:163
2995399129,2cyclictest6846-21head08:35:163
2995399129,2cyclictest6846-21head08:35:163
2995399129,2cyclictest5957-21wc08:35:013
2995399129,2cyclictest28758-21sed12:25:273
2995399129,2cyclictest22090-21tr08:00:153
2995399129,2cyclictest16324-21seq09:25:143
2995399129,2cyclictest15394-21grep11:20:143
2995399129,2cyclictest10981-21head07:35:213
2995399128,3cyclictest9646-21grep11:55:163
2995399128,2cyclictest29731-21sed08:15:173
2995399128,2cyclictest19018-21wc12:00:243
2995399128,2cyclictest17751-21uniq07:50:163
2995399127,4cyclictest537-21snmpd12:09:213
2995399124,4cyclictest225-21systemd-journal08:51:043
2995399120,1cyclictest0-21swapper/310:53:473
2995399120,11cyclictest0-21swapper/311:44:013
2994899129,2cyclictest32-21ksoftirqd/208:20:002
2994899129,2cyclictest28210-21sed08:10:252
2994899129,2cyclictest27093-21tr12:25:152
2994899129,2cyclictest2657-21mailstats08:25:222
2994899129,2cyclictest23812-21sed11:25:172
2994899129,2cyclictest1832-21grep07:15:262
2994899129,2cyclictest17747-21cron09:00:002
2994899128,3cyclictest4767-21chrt11:52:252
2994899128,3cyclictest10985-21uname07:35:212
2994899128,3cyclictest10647-21ls08:45:012
2994899125,3cyclictest0-21swapper/212:24:212
2994899124,3cyclictest32-21ksoftirqd/209:59:122
2994899120,1cyclictest0-21swapper/212:39:152
2994399129,2cyclictest3495-21head12:30:151
2994399129,2cyclictest14030-21head11:00:161
2994399128,3cyclictest32506-21ssh12:28:201
2994399128,3cyclictest30447-21tr07:10:141
2994399128,3cyclictest23727-21cut11:25:161
2994399128,3cyclictest17535-21tr12:00:121
2994399128,3cyclictest15346-21head11:20:141
2994399128,3cyclictest12812-21cut11:00:011
2994399127,4cyclictest5942-21cat07:25:201
2994399127,4cyclictest27553-21expr08:10:181
2994399122,6cyclictest19461-21aten2_r7power_c10:25:131
2994399121,9cyclictest6163-21aten2_r7power_p08:35:141
2994399121,9cyclictest6163-21aten2_r7power_p08:35:131
2994399120,2cyclictest0-21swapper/107:55:131
2993999129,2cyclictest28605-21cat12:25:260
2993999129,2cyclictest19729-21grep07:55:140
2993999129,2cyclictest17842-21cat09:00:010
2993999129,2cyclictest16783-21grep09:25:150
2993999128,3cyclictest16073-21expr11:40:140
2993999125,4cyclictest225-21systemd-journal08:30:010
2993999123,4cyclictest0-21swapper/010:30:210
29939991212,0cyclictest0-21swapper/012:37:110
29939991210,1cyclictest7887-21grep07:30:160
2993999120,2cyclictest0-21swapper/012:11:000
2995399118,2cyclictest9895-21ssh10:57:513
2995399118,2cyclictest6088-21cat11:15:113
2995399118,2cyclictest488-21ssh09:15:213
2995399118,2cyclictest348-21ssh10:32:473
2995399118,2cyclictest30010-21ssh10:11:293
2995399118,2cyclictest28547-21ssh09:51:153
2995399118,2cyclictest26909-21ssh09:30:523
2995399118,2cyclictest24116-21cut09:10:173
2995399118,2cyclictest23907-21ssh10:46:273
2995399118,2cyclictest23252-21pmu-power11:05:223
2995399118,2cyclictest21594-21grep07:59:593
2995399118,2cyclictest19328-21sh12:20:173
2995399118,2cyclictest16990-21ssh12:38:263
2995399118,2cyclictest15886-21tr08:55:143
2995399118,2cyclictest1505-21ssh11:50:193
2995399118,2cyclictest14246-21uname11:00:183
2995399110,10cyclictest22582-21chrt09:08:113
2994899118,2cyclictest7117-21ssh12:32:172
2994899118,2cyclictest3943-21tr08:30:182
2994899118,2cyclictest19682-21tr07:55:142
2994899118,2cyclictest15428-21head07:45:182
2994899118,2cyclictest12654-21expr10:40:162
2994899118,2cyclictest1213-21sed08:25:012
2994899114,4cyclictest18073-21grep09:00:132
2994399119,1cyclictest38150irq/131-snd_hda11:46:411
2994399119,1cyclictest19118-21grep10:05:191
2994399118,2cyclictest890-21expr07:15:171
2994399118,2cyclictest7458-21ssh09:20:071
2994399118,2cyclictest6994-21head11:15:171
2994399118,2cyclictest30993-21ssh10:12:141
2994399118,2cyclictest24510-21tr08:05:151
2994399118,2cyclictest20666-21ssh09:27:321
2994399118,2cyclictest18020-21basename09:00:111
2994399118,2cyclictest17795-21head07:50:171
2994399118,2cyclictest15506-21hwlatdetect07:45:181
2994399118,2cyclictest15374-21sh08:55:011
2994399118,2cyclictest14899-21ssh11:58:351
2994399118,2cyclictest12590-21ssh12:15:581
2994399118,2cyclictest12464-21tr10:40:151
2994399117,3cyclictest2833-21rm09:16:431
2994399116,2cyclictest4386-21taskset07:21:461
2994399111,1cyclictest141rcu_preempt11:05:561
2994399110,10cyclictest9181-21ssh12:14:501
2993999118,2cyclictest7031-21cat08:35:190
2993999118,2cyclictest7031-21cat08:35:180
2993999118,2cyclictest6569-21tail07:25:260
2993999118,2cyclictest3937-21tr08:30:170
2993999118,2cyclictest29242-21cut08:15:150
2993999118,2cyclictest28716-21basename08:15:010
2993999118,2cyclictest23258-21cat08:00:270
2993999118,2cyclictest21966-21tr11:05:150
2993999118,2cyclictest10218-21ssh10:19:490
2993999117,3cyclictest22733-21sh09:10:010
2993999110,8cyclictest0-21swapper/008:20:120
2993999110,2cyclictest0-21swapper/009:18:400
2993999110,10cyclictest11573-21sed07:35:260
2995399108,1cyclictest92952sleep307:33:513
2995399108,1cyclictest8433-21taskset08:40:093
2995399108,1cyclictest147002sleep307:45:143
2994899108,1cyclictest6216-21awk07:25:222
2994899108,1cyclictest22707-21chrt09:09:462
2994899106,3cyclictest17858-21cut07:50:172
2994899106,1cyclictest141rcu_preempt07:20:142
2994899100,9cyclictest11935-21grep08:45:192
2994399108,1cyclictest308832sleep108:17:331
2994399108,1cyclictest13089-21basename07:40:191
2994399107,2cyclictest23280-21ssh09:10:131
2994399107,2cyclictest20615-21basename09:05:121
2994399107,2cyclictest10909-21wget08:45:131
2994399107,2cyclictest10439-21chrt08:43:261
2994399105,1cyclictest225-21systemd-journal08:29:311
2994399102,8cyclictest0-21swapper/107:30:131
2993999108,1cyclictest16900-21ssh09:45:030
2993999108,1cyclictest11463-21grep08:45:160
2993999107,2cyclictest31304-21tr07:10:190
2993999107,2cyclictest25666-21cat08:05:260
2993999107,2cyclictest17047-21cat07:50:120
2993999106,3cyclictest15379-21uniq07:45:180
299539997,1cyclictest24171-21date08:05:003
299489997,1cyclictest6655-21dnf08:35:162
299489997,1cyclictest6655-21dnf08:35:152
299439997,1cyclictest22029-21egrep08:00:151
299399998,1cyclictest727-21fschecks_count07:15:170
299399998,1cyclictest2940-21cat07:20:130
299399996,2cyclictest9180-21grep08:40:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional