You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-19 - 22:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sun Apr 19, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
814628976,9sleep00-21swapper/007:08:580
812228158,19sleep30-21swapper/307:08:413
811028164,13sleep10-21swapper/107:08:311
798828168,9sleep20-21swapper/207:06:552
8332993028,1cyclictest0-21swapper/010:11:110
8332992928,1cyclictest0-21swapper/011:58:590
833899260,26cyclictest0-21swapper/209:53:382
833299267,5cyclictest141rcu_preempt10:50:010
833299260,24cyclictest0-21swapper/012:08:090
8338992424,0cyclictest0-21swapper/210:45:192
834299235,7cyclictest516-21gdbus11:25:233
833699230,22cyclictest0-21swapper/110:18:211
8336992221,1cyclictest0-21swapper/112:33:221
833699220,2cyclictest0-21swapper/108:00:201
833699220,22cyclictest0-21swapper/111:45:141
833699220,22cyclictest0-21swapper/110:54:011
8332992221,1cyclictest0-21swapper/010:34:530
833899210,21cyclictest0-21swapper/211:04:042
833899210,1cyclictest0-21swapper/210:10:462
833699210,21cyclictest0-21swapper/111:26:081
8338992018,1cyclictest0-21swapper/210:05:132
833899200,1cyclictest0-21swapper/212:35:142
8336992019,1cyclictest0-21swapper/112:05:311
8332992019,1cyclictest0-21swapper/011:45:340
834299190,18cyclictest0-21swapper/311:03:533
8338991918,1cyclictest0-21swapper/210:29:082
833699191,2cyclictest0-21swapper/110:40:111
834299180,17cyclictest0-21swapper/312:35:573
834299180,17cyclictest0-21swapper/312:34:383
8338991817,1cyclictest0-21swapper/211:31:482
833899180,18cyclictest0-21swapper/212:00:012
8336991816,1cyclictest0-21swapper/111:20:481
8336991815,3cyclictest0-21swapper/111:00:131
8336991811,1cyclictest0-21swapper/108:55:141
834299173,11cyclictest31130-21grep10:25:183
833899170,16cyclictest0-21swapper/211:10:272
833899170,16cyclictest0-21swapper/208:10:142
8336991715,1cyclictest22298-21apt-get10:40:001
833699170,16cyclictest0-21swapper/110:59:061
833299174,7cyclictest18728-21munin-run12:15:000
833299170,17cyclictest0-21swapper/011:01:540
834299160,16cyclictest0-21swapper/311:24:443
833899160,16cyclictest0-21swapper/211:56:472
833899160,16cyclictest0-21swapper/211:42:542
833899160,14cyclictest0-21swapper/212:32:202
833699165,5cyclictest0-21swapper/112:00:141
8336991616,0cyclictest0-21swapper/111:54:551
8336991616,0cyclictest0-21swapper/111:40:131
8332991614,1cyclictest0-21swapper/009:19:460
834299153,7cyclictest32246-21apache_accesses11:05:123
834299153,6cyclictest22304-21perf10:40:003
8338991514,1cyclictest0-21swapper/212:16:172
833899150,2cyclictest0-21swapper/209:31:412
833899150,15cyclictest0-21swapper/210:38:502
833899150,15cyclictest0-21swapper/209:42:012
833699152,12cyclictest0-21swapper/110:21:361
833699150,15cyclictest0-21swapper/111:12:291
833299152,7cyclictest9875-21awk09:15:000
833299151,1cyclictest141rcu_preempt09:54:480
833299150,7cyclictest5598-21awk10:30:000
250582150,0chrt0-21swapper/311:59:113
8342991412,1cyclictest0-21swapper/311:38:583
8342991411,2cyclictest14805-21ntpq10:15:203
8342991410,3cyclictest29545-21cut09:00:133
833899140,12cyclictest0-21swapper/211:35:102
833699147,2cyclictest0-21swapper/112:35:571
8336991413,1cyclictest0-21swapper/112:15:191
8336991412,2cyclictest0-21swapper/112:24:351
833699140,12cyclictest20127-21cat07:30:241
833299143,6cyclictest0-21swapper/010:20:130
8332991412,1cyclictest13-21ksoftirqd/010:05:020
8332991412,1cyclictest0-21swapper/012:03:000
833299140,4cyclictest25271-21cron11:40:000
833299140,14cyclictest0-21swapper/012:26:470
834299139,3cyclictest31280-21tune2fs07:55:163
834299131,11cyclictest22638-21unixbench_singl10:00:233
8342991310,2cyclictest5673-21cut10:10:163
8342991310,2cyclictest29210-21sleep09:00:013
8342991310,2cyclictest28941-21sed12:20:213
8342991310,2cyclictest2703-21cut09:10:153
8342991310,2cyclictest26035-21tr11:40:153
8342991310,2cyclictest21071-21grep07:35:143
8342991310,2cyclictest15296-21grep08:30:173
834299130,8cyclictest0-21swapper/310:05:133
834299130,13cyclictest0-21swapper/309:46:123
833899134,5cyclictest225-21systemd-journal12:10:012
833899132,1cyclictest1558-21diskmemload12:14:552
8338991310,2cyclictest32613-21sed09:05:162
833899130,2cyclictest9265-21kworker/u8:311:26:082
833899130,13cyclictest0-21swapper/210:52:132
833699139,3cyclictest22481-21cat10:00:231
833699137,5cyclictest38150irq/131-snd_hda10:05:131
8336991310,2cyclictest5440-21grep12:25:271
833699130,1cyclictest0-21swapper/109:59:341
833699130,0cyclictest0-21swapper/112:12:141
833299139,3cyclictest17370-21cut11:15:170
833299137,5cyclictest230322sleep007:39:590
833299136,4cyclictest11667-21apache212:38:400
833299135,2cyclictest225-21systemd-journal09:26:540
833299133,5cyclictest0-21swapper/008:35:150
833299132,11cyclictest0-21swapper/011:51:160
8332991311,1cyclictest0-21swapper/009:45:030
8332991310,2cyclictest25646-21cron07:44:590
834299129,2cyclictest9536-21sed11:10:213
834299129,2cyclictest9237-21wc08:15:213
834299129,2cyclictest31655-21grep10:45:143
834299129,2cyclictest3123-21cut08:05:163
834299129,2cyclictest27006-21tail07:45:213
834299129,2cyclictest26806-21cron08:55:003
834299129,2cyclictest2554-21cut11:45:183
834299129,2cyclictest23681-21cut07:40:143
834299129,2cyclictest18737-21ssh09:20:113
834299129,2cyclictest17810-21sed07:25:243
834299129,2cyclictest17807-21tr11:15:223
834299129,2cyclictest17635-21cut08:35:143
834299129,2cyclictest17563-21ssh10:55:253
834299129,2cyclictest1732-21cat08:00:193
834299129,2cyclictest12854-21tr08:25:133
834299129,2cyclictest12024-21head11:50:263
834299128,3cyclictest19544-21tr12:15:173
834299128,3cyclictest12533-21wc09:35:183
834299123,8cyclictest22915-21diskstats08:45:183
834299123,7cyclictest11572-21missed_timers09:15:193
834299122,7cyclictest2545-21wc09:30:003
834299120,11cyclictest19306-21ssh09:59:203
834299120,11cyclictest0-21swapper/309:31:583
833899129,2cyclictest3418-21cron09:50:012
833899129,2cyclictest21518-21head10:00:142
833899129,2cyclictest20136-21awk09:20:212
833899129,2cyclictest14920-21uname07:20:182
833899129,2cyclictest13185-21head09:55:192
833899124,8cyclictest32-21ksoftirqd/207:40:132
8338991212,0cyclictest0-21swapper/210:20:142
8338991211,1cyclictest55650irq/127-enp2s0-09:29:142
8338991210,1cyclictest22071-21ntpq07:35:202
833899120,3cyclictest0-21swapper/212:22:442
833899120,2cyclictest0-21swapper/211:52:242
833899120,1cyclictest0-21swapper/208:25:132
833899120,11cyclictest32549-21grep11:05:152
833699129,2cyclictest9523-21head07:10:171
833699129,2cyclictest28751-21grep09:45:181
833699121,9cyclictest0-21swapper/109:18:071
8336991211,1cyclictest0-21swapper/111:37:011
8336991210,1cyclictest0-21swapper/110:31:561
833699120,1cyclictest0-21swapper/111:34:561
833699120,11cyclictest24740-21grep08:50:131
833699120,11cyclictest0-21swapper/107:45:141
833299129,2cyclictest9679-21head08:15:250
833299129,2cyclictest9392-21grep11:30:160
833299129,2cyclictest29278-21wc07:50:200
833299129,2cyclictest27314-21tr08:55:140
833299129,2cyclictest25835-21ssh11:20:200
833299129,2cyclictest20778-21chrt10:38:510
833299129,2cyclictest15918-21tail08:30:180
833299129,2cyclictest12609-21tr09:55:150
833299129,1cyclictest0-21swapper/010:17:420
833299128,3cyclictest31696-21wc09:05:000
833299128,3cyclictest28254-21uniq12:20:160
833299128,3cyclictest11423-21tr07:15:150
833299124,4cyclictest225-21systemd-journal08:25:000
833299121,6cyclictest23412-21munin-run11:00:000
8332991210,1cyclictest31858-21sed07:55:200
8332991210,1cyclictest22625-21head10:00:240
8332991210,1cyclictest13-21ksoftirqd/009:40:420
833299120,1cyclictest0-21swapper/012:32:520
834299119,1cyclictest23334-21processes10:20:233
834299118,2cyclictest9907-21grep11:30:183
834299118,2cyclictest5854-21tr08:10:153
834299118,2cyclictest5178-21ssh09:50:193
834299118,2cyclictest500-21sed09:05:203
834299118,2cyclictest3321-21rm12:25:083
834299118,2cyclictest2701-21tr12:05:143
834299118,2cyclictest22138-21wc08:45:003
834299118,2cyclictest19884-21grep09:40:133
834299118,2cyclictest11121-21cut12:10:143
834299118,2cyclictest10995-21taskset10:51:483
834299117,3cyclictest6369-21tr10:30:163
834299113,4cyclictest225-21systemd-journal08:25:013
834299110,10cyclictest30662-21sh12:02:023
834299110,10cyclictest26931-21ssh10:41:553
833899119,1cyclictest3754-21grep08:05:182
833899119,1cyclictest25470-21df11:20:162
833899119,1cyclictest19694-21chrt07:30:192
833899118,2cyclictest5852-21sh10:30:042
833899118,2cyclictest3822-21cat09:10:242
833899118,2cyclictest3233-21ssh12:25:032
833899118,2cyclictest16439-21tr07:25:142
833899118,2cyclictest13944-21ssh09:35:522
833899113,6cyclictest0-21swapper/211:16:272
833899110,10cyclictest0-21swapper/210:58:252
833699119,1cyclictest8764-21grep08:15:161
833699119,1cyclictest32073-21grep09:05:141
833699119,1cyclictest11147-21head08:20:171
833699118,3cyclictest1374-21ssh09:29:081
833699118,3cyclictest13346-21ssh09:35:261
833699118,2cyclictest5377-21expr10:10:151
833699118,2cyclictest4119-21cat09:30:191
833699118,2cyclictest321-21cat08:00:001
833699118,2cyclictest30069-21ssh10:25:081
833699118,2cyclictest23801-21tr07:40:141
833699118,2cyclictest23057-21grep07:40:001
833699118,2cyclictest19023-21tr09:20:131
833699118,2cyclictest18316-21head08:35:171
833699118,2cyclictest17312-21tail11:15:171
833699118,2cyclictest15471-21tr08:30:161
833699118,2cyclictest13648-21basename08:25:171
833699118,2cyclictest11173-21wget07:15:131
8336991110,1cyclictest1558-21diskmemload11:57:161
833699110,10cyclictest28913-21cut07:50:161
833299119,1cyclictest9004-21sed11:10:160
833299118,2cyclictest9483-21grep07:10:160
833299118,2cyclictest9235-21head10:50:260
833299118,2cyclictest502-21ssh11:25:030
833299118,2cyclictest3755-21ssh11:06:560
833299118,2cyclictest30215-21ssh11:42:200
833299118,2cyclictest27931-21ssh10:42:430
833299118,2cyclictest26820-21apt08:55:000
833299118,2cyclictest19041-21tr09:20:130
833299118,2cyclictest14455-21grep07:20:150
833299118,2cyclictest13051-21wc09:35:250
833299117,3cyclictest18443-21wc07:30:010
833299114,6cyclictest22089-21basename08:45:000
833299110,10cyclictest23144-21sh12:16:580
834299108,1cyclictest29683-21python07:50:243
834299108,1cyclictest15165-21sendmail_mailtr07:20:213
834299107,2cyclictest9550-21chrt07:10:173
834299107,2cyclictest19291-21tail07:30:153
834299107,2cyclictest11431-21cut07:15:143
833899108,2cyclictest31453-21taskset09:03:312
833899108,2cyclictest311rcuc/208:45:222
833899108,2cyclictest28426-21wget07:50:142
833899108,2cyclictest14518-21sh09:17:032
833899108,1cyclictest9973-21ls07:10:222
833899108,1cyclictest26636-21sshd08:52:532
833899108,1cyclictest2116-21expr11:45:162
833899107,2cyclictest31387-21grep07:55:162
833899107,2cyclictest12425-21grep07:15:212
833899101,5cyclictest24463-21ps10:40:212
833899100,9cyclictest26275-21expr07:45:162
833899100,9cyclictest13521-21taskset10:15:122
833699108,2cyclictest4230-21taskset08:05:231
833699108,2cyclictest31032-21cat09:00:251
833699108,2cyclictest15614-21chrt07:21:121
833699108,1cyclictest22370-21wget08:45:171
833699108,1cyclictest17412-21awk07:25:201
833699107,2cyclictest7243-21grep08:10:271
833699107,2cyclictest4235-21cut09:50:141
833699107,2cyclictest31858-21ssh10:45:161
833699107,2cyclictest20782-21cat09:40:181
833699107,2cyclictest1769-21ssh09:10:021
833699107,2cyclictest1089-21cat11:05:201
833299108,1cyclictest7596-21chrt08:13:440
833299108,1cyclictest4119-21grep08:05:210
833299108,1cyclictest32118-21cut09:05:140
833299108,1cyclictest26938-21chrt07:45:190
833299107,2cyclictest867-21tr08:00:150
833299107,2cyclictest4963-21grep09:30:260
833299107,2cyclictest24390-21taskset08:49:330
833299107,2cyclictest20249-21grep07:30:250
833299107,2cyclictest13979-21ntpq08:25:220
83389998,1cyclictest9806-21taskset08:16:302
83389998,1cyclictest857-21expr08:00:142
83389997,2cyclictest28285-21cut08:55:222
83389997,2cyclictest19931-21cat08:40:102
83389997,2cyclictest15306-21basename08:30:162
83389997,1cyclictest118302sleep208:20:242
83389995,3cyclictest19225-21unixbench_multi08:35:262
83369992,1cyclictest225-21systemd-journal08:45:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional