You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-02 - 23:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Fri Jan 02, 2026 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1054529166,9sleep20-21swapper/207:06:302
1053828977,8sleep00-21swapper/007:06:240
1011528974,10sleep30-21swapper/307:05:273
1051728767,15sleep10-21swapper/107:06:091
10894991715,1cyclictest0-21swapper/110:10:141
1089099170,2cyclictest0-21swapper/010:45:130
1090499162,5cyclictest225-21systemd-journal12:25:003
1090499160,2cyclictest0-21swapper/307:30:273
1089099163,7cyclictest21025-21idleruntime-cro12:00:000
10904991511,2cyclictest17432-21grep08:30:133
1090199151,12cyclictest30752-21cat10:05:012
1090199150,12cyclictest0-21swapper/209:55:132
10894991513,1cyclictest0-21swapper/107:50:211
1089499150,15cyclictest24684-21latency12:05:191
1089099153,6cyclictest13897-21idleruntime-cro11:45:010
1089099151,8cyclictest9889-21munin-run08:15:000
1089099151,8cyclictest18464-21sendmail-msp09:40:000
1089099151,8cyclictest18464-21sendmail-msp09:40:000
1090499141,3cyclictest0-21swapper/308:05:133
10904991411,2cyclictest39-21ksoftirqd/311:20:203
10904991410,3cyclictest30375-21wc07:50:003
1090499140,2cyclictest0-21swapper/309:20:143
10901991411,2cyclictest19372-21tail10:45:282
10901991411,2cyclictest13976-21wc10:35:212
10901991410,3cyclictest7002-21grep08:05:262
10901991410,3cyclictest20631-21sort08:35:172
10901991410,3cyclictest11677-21grep10:30:212
10901991410,2cyclictest10724-21wc09:20:262
1090199140,13cyclictest9212-21wc08:10:242
10894991411,2cyclictest22276-21grep07:30:291
10894991411,2cyclictest17447-21ls10:45:001
10890991410,3cyclictest6043-21uniq08:05:170
10890991410,3cyclictest21017-21cron09:45:000
1090499139,2cyclictest24794-21cat11:00:003
1090499139,2cyclictest18124-21sed11:50:273
1090499132,6cyclictest39-21ksoftirqd/310:35:003
1090499131,1cyclictest141rcu_preempt10:50:133
1090499131,1cyclictest141rcu_preempt10:45:173
10904991310,2cyclictest6036-21tr10:20:143
10904991310,2cyclictest29220-21tail10:00:153
10904991310,2cyclictest22168-21grep08:40:133
10904991310,2cyclictest17018-21grep07:20:203
10904991310,2cyclictest15615-21cut10:40:153
1090199139,3cyclictest30238-21wc08:55:212
1090199135,6cyclictest1-21systemd09:13:312
10901991310,2cyclictest8183-21grep09:15:272
10901991310,2cyclictest534-21cron10:10:002
10901991310,2cyclictest32335-21grep11:15:172
10901991310,2cyclictest31208-21grep07:50:242
10901991310,2cyclictest27017-21cron11:05:002
10901991310,2cyclictest26810-21grep12:10:182
10901991310,2cyclictest20592-21grep09:40:272
10901991310,2cyclictest15171-21wc11:45:222
10901991310,2cyclictest1315-21grep12:25:142
1089499139,3cyclictest19972-21sed07:25:261
10894991310,2cyclictest5312-21turbostat.cron10:19:591
10894991310,2cyclictest31776-21sh11:15:001
10894991310,2cyclictest28197-21dump-pmu-power12:15:001
10894991310,2cyclictest25839-21ls09:55:001
10894991310,2cyclictest22750-21tr10:55:141
10894991310,2cyclictest18942-21grep08:30:261
10894991310,2cyclictest17044-21awk08:30:001
10894991310,2cyclictest17025-21tail09:35:161
10894991310,2cyclictest17025-21tail09:35:161
1089099139,3cyclictest32311-21grep11:15:170
1089099131,6cyclictest18639-21awk11:55:000
10890991313,0cyclictest0-21swapper/012:05:120
10890991310,2cyclictest4982-21grep09:10:210
10890991310,2cyclictest2951-21sleep10:15:000
10890991310,2cyclictest28006-21tr08:50:210
10890991310,2cyclictest15792-21grep11:45:290
10890991310,2cyclictest10182-21grep08:15:180
1089099130,12cyclictest7731-21cat11:30:200
1089099130,12cyclictest7731-21cat11:30:200
1089099130,12cyclictest13223-21tr10:35:150
1090499129,2cyclictest9634-21tr11:35:153
1090499129,2cyclictest4660-21chrt12:30:223
1090499129,2cyclictest402-21head09:00:223
1090499129,2cyclictest26135-21cut09:55:133
1090499129,2cyclictest23703-21grep09:50:143
1090499129,2cyclictest2243-21grep07:55:253
1090499129,2cyclictest21549-21tr09:45:153
1090499129,2cyclictest11827-21tr09:25:163
1090499129,2cyclictest10382-21tr08:15:193
1090499124,6cyclictest2053-21/usr/sbin/munin12:25:203
10904991210,1cyclictest28973-21cron08:55:003
1090499120,1cyclictest141rcu_preempt08:00:253
1090199129,2cyclictest9381-21sed10:25:192
1090199129,2cyclictest28075-21grep11:05:172
1090199129,2cyclictest27649-21head08:50:182
1090199129,2cyclictest24824-21head07:35:332
1090199129,2cyclictest22805-21wc09:45:282
1090199129,2cyclictest19175-21grep11:55:162
1090199129,2cyclictest18323-21tr07:25:132
1090199129,2cyclictest1824-21tr09:05:132
1090199129,2cyclictest13686-21basename09:30:002
1090199129,2cyclictest13307-21wc11:40:262
1090199129,2cyclictest13279-21grep08:20:252
1090199129,2cyclictest10218-21tr08:15:192
1090199128,3cyclictest17882-21wc09:35:252
1090199128,3cyclictest17882-21wc09:35:252
1090199123,8cyclictest14539-21memory07:15:212
1089499129,2cyclictest5883-21cut12:35:131
1089499129,2cyclictest31125-21cut12:20:151
1089499129,2cyclictest29509-21tr08:55:161
1089499129,2cyclictest21547-21grep09:45:151
1089499129,2cyclictest19651-21cut11:55:181
1089499129,2cyclictest17188-21grep11:50:171
1089499129,2cyclictest1581-21df12:25:151
1089499129,2cyclictest15126-21grep07:15:271
1089499128,3cyclictest17778-21taskset07:22:031
1089499128,3cyclictest11309-21grep10:30:181
1089499127,4cyclictest12792-21grep07:10:271
1089499124,2cyclictest225-21systemd-journal11:34:011
1089499124,2cyclictest225-21systemd-journal11:34:001
1089499122,9cyclictest31769-21munin-node10:05:361
1089499121,9cyclictest20705-21kworker/1:012:00:201
10894991210,1cyclictest6351-21cat08:05:191
10894991210,1cyclictest19737-21grep08:35:121
1089099129,2cyclictest3426-21grep10:15:160
1089099129,2cyclictest29924-21grep08:55:180
1089099129,2cyclictest29875-21cut10:00:210
1089099129,2cyclictest28198-21cron10:00:000
1089099129,2cyclictest25143-21grep08:45:170
1089099129,2cyclictest21305-21ls10:50:220
1089099129,2cyclictest16650-21grep07:20:170
1089099128,3cyclictest9912-21grep09:20:160
1089099128,3cyclictest7976-21wc12:40:000
1089099128,3cyclictest3693-21cut08:00:160
1089099128,3cyclictest29498-21cat07:45:280
1089099128,2cyclictest4539-21tr11:25:140
1089099127,3cyclictest11506-21tr11:40:000
1089099125,3cyclictest225-21systemd-journal07:29:550
1089099121,6cyclictest27045-21perf11:05:000
1089099120,2cyclictest0-21swapper/009:00:130
1090499119,1cyclictest203402sleep307:29:273
1090499118,2cyclictest8806-21taskset11:34:113
1090499118,2cyclictest8806-21taskset11:34:113
1090499118,2cyclictest7894-21sendmail12:40:003
1090499118,2cyclictest3829-21cut10:15:183
1090499118,2cyclictest32412-21cat11:15:173
1090499118,2cyclictest31140-21cut10:05:153
1090499118,2cyclictest30705-21sort11:10:213
1090499118,2cyclictest3046-21wc09:05:243
1090499118,2cyclictest28662-21tr11:05:233
1090499118,2cyclictest27717-21taskset07:43:563
1090499118,2cyclictest25025-21awk11:00:133
1090499118,2cyclictest20916-21taskset09:43:453
1090499118,2cyclictest20316-21cut08:35:143
1090499118,2cyclictest13814-21wc07:15:153
1090499118,2cyclictest13192-21grep10:35:153
1090199118,2cyclictest953-21grep10:10:142
1090199118,2cyclictest7062-21tr11:30:152
1090199118,2cyclictest7062-21tr11:30:152
1090199118,2cyclictest6558-21grep10:20:162
1090199118,2cyclictest4862-21tail10:15:292
1090199118,2cyclictest3804-21taskset11:20:562
1090199118,2cyclictest3635-21grep12:30:142
1090199118,2cyclictest30034-21grep12:15:272
1090199118,2cyclictest26792-21grep07:40:202
1090199118,2cyclictest25269-21grep08:45:182
1090199118,2cyclictest22693-21taskset07:34:102
1090199118,2cyclictest1962-21tr07:55:222
1090199118,2cyclictest11882-21grep07:10:182
1090199117,3cyclictest15631-21tr10:40:152
1090199117,2cyclictest23792-21grep12:05:132
1090199117,2cyclictest20373-21tr10:50:152
1090199114,3cyclictest225-21systemd-journal07:49:102
1090199110,10cyclictest9387-21cut11:35:142
1090199110,10cyclictest15830-21chrt09:32:232
1089499119,1cyclictest6296-21cron09:15:001
1089499119,1cyclictest5850-21tr10:20:131
1089499119,1cyclictest3255-21grep11:20:221
1089499119,1cyclictest14260-21grep08:20:351
1089499118,2cyclictest9794-21expr11:35:161
1089499118,2cyclictest8685-21ls09:20:001
1089499118,2cyclictest8517-21tail08:10:151
1089499118,2cyclictest3282-21grep08:00:141
1089499118,2cyclictest25872-21cat11:00:181
1089499118,2cyclictest25283-21grep08:45:181
1089499118,2cyclictest20902-21wc10:50:191
1089499118,2cyclictest13468-21grep11:40:271
1089499118,2cyclictest10279-21sed09:20:191
1089499118,2cyclictest10238-21aten2_r7power_p08:15:191
1089499117,4cyclictest28765-21grep11:05:261
1089499117,3cyclictest6457-21taskset11:29:331
1089499117,1cyclictest30244-21taskset12:15:571
1089099118,2cyclictest8913-21cat09:20:000
1089099118,2cyclictest6204-21expr10:20:140
1089099118,2cyclictest30377-21grep11:10:180
1089099118,2cyclictest26381-21sort07:40:160
1089099118,2cyclictest24532-21taskset10:58:580
1089099118,2cyclictest23969-21cat09:50:150
1089099118,2cyclictest21535-21grep09:45:160
1089099118,2cyclictest20343-21cut08:35:140
1089099118,2cyclictest18743-21ls08:30:250
1089099118,2cyclictest13318-21head08:20:250
1089099118,2cyclictest13193-21grep09:25:280
1089099118,2cyclictest13036-21chrt07:12:300
1089099116,2cyclictest225-21systemd-journal10:35:000
1089099114,6cyclictest31769-21munin-node12:00:490
1089099114,3cyclictest225-21systemd-journal12:30:000
1089099110,10cyclictest23189-21chrt07:35:210
1089099110,10cyclictest15351-21aten2_r7power_c10:40:140
1090499108,1cyclictest7506-21sed09:15:193
1090499108,1cyclictest5901-21sed11:25:273
1090499108,1cyclictest30584-21cut07:50:213
1090499108,1cyclictest304672sleep312:18:483
1090499108,1cyclictest29780-21sed08:55:173
1090499108,1cyclictest26167-21aten2_r7power_p12:10:153
1090499108,1cyclictest14098-21taskset11:45:053
1090499107,2cyclictest5301-21grep09:10:233
1090499107,2cyclictest23006-21chrt07:35:123
1090499107,2cyclictest20209-21fgrep11:55:243
1090499107,2cyclictest17415-21grep09:35:193
1090499107,2cyclictest17415-21grep09:35:193
1090499106,3cyclictest15150-21cut09:30:233
1090499105,2cyclictest225-21systemd-journal07:10:103
1090499104,1cyclictest141rcu_preempt12:00:133
1090499100,9cyclictest0-21swapper/308:45:143
1090499100,2cyclictest225-21systemd-journal08:23:223
1090499100,1cyclictest0-21swapper/310:27:413
1090199108,1cyclictest5837-21sed11:25:262
1090199108,1cyclictest573-21cat09:00:232
1090199108,1cyclictest44062sleep208:00:242
1090199108,1cyclictest23590-21grep10:55:202
1090199108,1cyclictest230912sleep212:01:202
1090199108,1cyclictest17310-21cut08:30:132
1090199108,1cyclictest17116-21cut07:20:232
1090199108,1cyclictest16590-21chrt11:50:152
1090199107,2cyclictest6045-21tr12:35:132
1090199107,2cyclictest30924-21proc_pri11:10:242
1090199107,2cyclictest24155-21expr09:50:152
1090199104,4cyclictest17019-21cron08:29:592
1090199100,9cyclictest32709-21taskset12:21:472
1089499108,2cyclictest14366-21ls10:35:261
1089499108,1cyclictest32110-21hostname11:15:161
1089499108,1cyclictest28736-21cut10:00:121
1089499108,1cyclictest28548-21grep08:50:281
1089499108,1cyclictest22382-21cat08:40:141
1089499108,1cyclictest1753-21ntpq07:55:191
1089499108,1cyclictest160582sleep111:47:451
1089499108,1cyclictest1336-21apt09:05:001
1089499107,2cyclictest2826-21head09:05:201
1089499107,2cyclictest26099-21expr07:40:141
1089499107,2cyclictest24427-21find07:35:301
1089499107,2cyclictest18360-21fschecks_time10:45:171
1089499107,2cyclictest14961-21taskset09:30:201
1089499107,2cyclictest13259-21grep09:25:281
1089499106,1cyclictest8147-21taskset10:25:061
1089499100,9cyclictest208322sleep109:42:391
1089099108,2cyclictest3330-21rt-features11:20:230
1089099108,1cyclictest8181-21chrt10:25:100
1089099108,1cyclictest3786-21cat12:30:140
1089099108,1cyclictest31469-21fschecks_count10:05:170
1089099108,1cyclictest27743-21expr11:05:150
1089099108,1cyclictest27235-21ntpq12:10:220
1089099108,1cyclictest13649-21hostname07:15:150
1089099107,2cyclictest30502-21grep07:50:210
1089099107,2cyclictest29681-21ls12:15:230
1089099107,2cyclictest2666-21cat09:05:180
1089099107,2cyclictest22475-21expr08:40:140
1089099107,2cyclictest21118-21cat07:30:170
1089099107,2cyclictest16584-21grep08:25:260
1089099107,2cyclictest14029-21aten2_r7power_p09:30:140
1089099102,7cyclictest606-21apache_processe07:55:110
1089099100,9cyclictest30873-21aten2_r7power_c12:20:150
109049997,1cyclictest39-21ksoftirqd/308:10:123
109049996,2cyclictest11784-21cut11:40:133
109049995,1cyclictest39-21ksoftirqd/308:25:133
109049994,4cyclictest39-21ksoftirqd/312:05:263
109049990,6cyclictest0-21swapper/310:10:133
109019997,1cyclictest22063-21apache_processe08:40:112
108949997,2cyclictest4833-21sed12:30:261
108949997,1cyclictest28537-21expr07:45:171
108949997,1cyclictest26144-21basename09:55:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional