You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-06 - 21:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Tue Jan 06, 2026 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1441229178,9sleep20-21swapper/207:08:352
1425129165,8sleep00-21swapper/007:06:290
1437128366,13sleep10-21swapper/107:08:031
1433128067,9sleep30-21swapper/307:07:333
1460399312,26cyclictest22426-21apt-get07:25:142
14601992625,1cyclictest0-21swapper/010:03:470
1460299240,22cyclictest0-21swapper/111:00:141
1460399232,3cyclictest10696-21idleruntime-cro11:30:002
1460199238,6cyclictest225-21systemd-journal11:20:000
1460799222,14cyclictest0-21swapper/307:50:143
1460799220,7cyclictest0-21swapper/310:40:133
1460399222,3cyclictest28456-21pmu-power07:35:222
1460299220,21cyclictest0-21swapper/107:40:101
1460199227,7cyclictest30748-21perf11:04:590
1460399214,2cyclictest23791-21apt-get10:50:132
1460399202,1cyclictest0-21swapper/210:35:142
1460399201,4cyclictest30574-21cat09:55:192
1460399201,3cyclictest2733-21cut07:50:182
1460399201,3cyclictest10710-21df_abs12:35:152
1460399194,13cyclictest0-21swapper/211:40:232
1460399193,3cyclictest3632-21processes10:05:222
1460299182,15cyclictest25779-21apt-get08:40:151
1460299180,18cyclictest0-21swapper/108:45:231
1460199180,3cyclictest0-21swapper/009:05:130
1460399171,3cyclictest6793-21cut08:00:132
1460399171,3cyclictest12552-21sed11:30:262
14603991713,3cyclictest12922-21ls10:25:202
14603991712,2cyclictest0-21swapper/209:00:142
1460799167,3cyclictest225-21systemd-journal09:58:563
1460399169,3cyclictest30857-21grep07:40:202
1460399163,3cyclictest21379-21grep11:50:162
1460199167,8cyclictest4087-21perf07:55:010
1460199161,9cyclictest10698-21perf11:30:000
1460399159,5cyclictest27069-21grep12:00:252
1460399158,6cyclictest5820-21grep12:25:152
1460399158,6cyclictest20014-21taskset07:20:112
1460399158,6cyclictest14935-21wc09:25:012
1460399151,3cyclictest553-21sshd09:26:372
1460399151,3cyclictest553-21sshd09:26:372
14603991510,4cyclictest1071-21grep11:10:122
14607991410,2cyclictest14332-21grep10:30:143
1460399149,4cyclictest32725-21head07:45:162
1460399149,4cyclictest26451-21tr10:55:172
1460399148,5cyclictest28232-21cut08:45:142
1460399148,5cyclictest19534-21head11:45:202
1460399148,5cyclictest18038-21cut09:30:152
1460399148,5cyclictest15767-21taskset08:17:362
1460399148,5cyclictest10895-21cat09:15:172
1460399147,6cyclictest9039-21chrt10:17:442
1460399147,6cyclictest3959-21cat11:15:132
1460399147,6cyclictest21667-21expr10:45:142
1460399147,6cyclictest19835-21smartctl10:40:172
1460399147,6cyclictest17957-21head08:20:252
1460399147,6cyclictest11143-21taskset08:09:202
1460399146,2cyclictest5454-21expr10:10:182
1460399144,9cyclictest15450-21dump-pmu-power11:40:012
1460399144,5cyclictest26895-21cron07:35:002
1460399144,5cyclictest26895-21cron07:34:592
1460399143,3cyclictest225-21systemd-journal09:40:562
1460399142,2cyclictest225-21systemd-journal12:20:002
1460399141,3cyclictest225-21systemd-journal08:15:002
1460399141,2cyclictest225-21systemd-journal12:10:002
1460399141,2cyclictest225-21systemd-journal07:18:342
1460299148,5cyclictest7416-21idleruntime-cro09:10:001
14602991413,1cyclictest0-21swapper/109:55:201
14602991411,2cyclictest8046-21chrt10:15:181
1460199144,5cyclictest20419-21munin-run11:50:010
1460199141,12cyclictest0-21swapper/008:45:140
14601991410,2cyclictest21106-21grep08:30:140
1460799139,3cyclictest23084-21sed10:45:263
1460799139,2cyclictest23186-21head07:25:183
1460799139,2cyclictest12160-21tr10:25:153
14607991310,2cyclictest6382-21sed10:10:273
14607991310,2cyclictest29105-21cron09:55:013
14607991310,2cyclictest28853-21grep07:35:263
14607991310,2cyclictest26705-21wc12:00:213
14607991310,2cyclictest16136-21wc08:20:003
1460399139,3cyclictest1180-21sort08:55:172
1460399138,4cyclictest6249-21taskset07:57:452
1460399138,4cyclictest25279-21taskset08:37:322
1460399138,4cyclictest19846-21basename09:35:152
1460399138,4cyclictest14868-21fschecks_count10:30:152
1460399138,4cyclictest11532-21chrt10:24:012
1460399137,5cyclictest9615-21chrt12:31:302
1460399137,5cyclictest7151-21taskset09:06:452
1460399137,5cyclictest26703-21uname08:40:212
1460399137,5cyclictest23012-21sshd11:55:062
1460399136,6cyclictest874-21uname10:00:202
1460399135,2cyclictest19427-21hostname08:25:172
1460399132,3cyclictest225-21systemd-journal09:55:002
1460399132,2cyclictest225-21systemd-journal11:04:202
1460399132,2cyclictest225-21systemd-journal07:13:372
14603991310,2cyclictest22695-21sed08:30:262
1460399130,4cyclictest225-21systemd-journal09:13:582
1460399130,2cyclictest0-21swapper/211:05:122
1460299139,3cyclictest24014-21grep10:50:141
1460299135,6cyclictest1-21systemd10:59:371
14602991310,2cyclictest7159-21grep11:20:221
14602991310,2cyclictest7159-21grep11:20:211
14602991310,2cyclictest25940-21grep12:00:141
14602991310,2cyclictest24108-21tail08:35:151
14602991310,2cyclictest19030-21grep09:30:251
14602991310,2cyclictest17371-21rm07:15:001
14602991310,2cyclictest17367-21head10:35:171
14602991310,2cyclictest1675-21grep11:10:161
1460299130,12cyclictest17991-21wc11:45:011
1460199139,3cyclictest25822-21cat07:30:180
1460199139,3cyclictest25822-21cat07:30:180
1460199139,3cyclictest19396-21grep08:25:170
1460199134,8cyclictest3079-21idleruntime-cro11:15:000
14601991310,2cyclictest8892-21grep12:30:210
14601991310,2cyclictest2225-21grep08:55:260
14601991310,2cyclictest15799-21grep07:10:170
1460799129,2cyclictest821-21wc07:45:213
1460799129,2cyclictest4272-21head11:15:153
1460799129,2cyclictest30398-21tr12:10:143
1460799129,2cyclictest27394-21sed08:40:283
1460799129,2cyclictest21146-21grep07:20:213
1460799129,2cyclictest1822-21grep12:15:253
1460799129,2cyclictest1690-21grep11:10:153
1460799129,2cyclictest14994-21sed11:35:253
1460799129,2cyclictest10174-21grep11:25:273
1460799128,3cyclictest18654-21grep09:30:203
1460799128,2cyclictest10305-21tr12:35:143
1460799127,2cyclictest19783-21systemctl07:20:003
1460399129,2cyclictest3363-21sed12:20:172
1460399129,2cyclictest25286-21grep09:45:162
1460399126,1cyclictest7713-21chrt11:20:542
1460399126,1cyclictest7713-21chrt11:20:532
1460399121,10cyclictest0-21swapper/212:10:142
1460299129,2cyclictest7431-21tail08:00:161
1460299129,2cyclictest32604-21grep11:05:241
1460299129,2cyclictest3145-21sed10:05:161
1460299129,2cyclictest23909-21grep09:40:261
1460299129,2cyclictest20307-21tr07:20:161
1460299129,2cyclictest19302-21grep08:25:161
1460299129,2cyclictest18369-21turbostat08:25:001
1460299129,2cyclictest13547-21grep11:35:141
1460299129,2cyclictest11213-21grep10:20:251
1460299129,2cyclictest10839-21grep12:35:171
1460299128,3cyclictest2024-21grep07:50:151
1460299128,2cyclictest8705-21grep11:25:151
1460299120,11cyclictest5026-21tr10:10:151
1460299120,11cyclictest18403-21cut07:15:161
1460199129,2cyclictest32280-21grep07:45:140
1460199129,2cyclictest29761-21grep07:40:120
1460199129,2cyclictest29550-21ls07:40:010
1460199129,2cyclictest2839-21tr12:20:150
1460199129,2cyclictest27188-21grep09:50:130
1460199129,2cyclictest26554-21head08:40:190
1460199129,2cyclictest24298-21cron09:45:000
1460199129,2cyclictest23125-21grep07:25:180
1460199129,2cyclictest21271-21grep07:20:210
1460199129,2cyclictest18290-21grep10:35:270
1460199129,2cyclictest12893-21grep09:20:150
1460199129,2cyclictest12123-21grep10:25:150
1460199129,2cyclictest11203-21grep10:20:250
1460199128,3cyclictest24513-21head10:50:170
1460199128,3cyclictest21144-21sh10:45:000
1460199128,3cyclictest12362-21cat12:40:000
1460199128,2cyclictest7130-21sed11:20:220
1460199128,2cyclictest7130-21sed11:20:220
1460199126,2cyclictest225-21systemd-journal08:20:000
1460199121,9cyclictest0-21swapper/011:40:140
1460199120,9cyclictest0-21swapper/009:00:130
1460199120,11cyclictest14409-21cat11:35:180
1460799118,2cyclictest7114-21apache_processe10:15:113
1460799118,2cyclictest6066-21head09:05:173
1460799118,2cyclictest30324-21munin-run08:50:003
1460799118,2cyclictest28983-21grep12:05:193
1460799118,2cyclictest2860-21sed09:00:003
1460799118,2cyclictest25399-21grep10:50:263
1460799118,2cyclictest24979-21taskset09:45:143
1460799118,2cyclictest23451-21wc09:40:223
1460799118,2cyclictest22628-21chrt11:53:093
1460799118,2cyclictest2240-21gpgconf10:05:003
1460799118,2cyclictest17408-21sed10:35:173
1460799118,2cyclictest14649-21taskset09:23:243
1460799118,2cyclictest12251-21grep08:10:173
1460799118,2cyclictest10853-21taskset08:05:343
1460799112,7cyclictest17556-21processes08:20:223
1460399110,3cyclictest225-21systemd-journal08:53:542
1460299119,1cyclictest8025-21tr12:30:151
1460299119,1cyclictest25-21ksoftirqd/107:55:221
1460299119,1cyclictest0-21swapper/110:30:111
1460299118,2cyclictest792-21grep08:55:151
1460299118,2cyclictest5455-21taskset11:17:401
1460299118,2cyclictest4752-21cat12:25:001
1460299118,2cyclictest3717-21head09:00:161
1460299118,2cyclictest32169-21cat10:00:121
1460299118,2cyclictest14685-21basename08:15:181
1460299118,2cyclictest12987-21ls09:20:151
1460299118,2cyclictest10964-21taskset08:07:011
1460299117,3cyclictest32680-21cron08:55:001
1460299117,3cyclictest28889-21sed07:35:261
1460299117,3cyclictest23259-21wc08:35:001
1460299117,3cyclictest22542-21tr10:45:191
1460299115,2cyclictest24603-21ls07:30:001
1460299113,4cyclictest225-21systemd-journal12:09:331
1460299113,4cyclictest225-21systemd-journal09:40:001
1460199118,2cyclictest7834-21grep10:15:160
1460199118,2cyclictest28372-21sendmail-msp11:00:000
1460199118,2cyclictest24697-21basename09:45:140
1460199118,2cyclictest22150-21head10:45:160
1460199118,2cyclictest21009-21cut09:35:220
1460199118,2cyclictest17224-21cat08:20:190
1460199118,2cyclictest15107-21cut09:25:150
1460199118,2cyclictest15107-21cut09:25:140
1460199118,2cyclictest13048-21idleruntime-cro11:35:000
1460199118,2cyclictest0-21swapper/012:00:140
1460199117,3cyclictest17937-21tr07:15:140
1460799108,1cyclictest7168-21expr08:00:143
1460799108,1cyclictest5903-21chrt11:20:113
1460799108,1cyclictest5903-21chrt11:20:113
1460799108,1cyclictest5544-21expr12:25:143
1460799108,1cyclictest45042sleep312:22:223
1460799108,1cyclictest323932sleep308:51:223
1460799108,1cyclictest31548-21chrt07:42:463
1460799108,1cyclictest31246-21grep11:05:133
1460799108,1cyclictest24832-21taskset11:55:553
1460799108,1cyclictest11210-21ntp_states09:15:213
1460799107,2cyclictest8417-21head09:10:163
1460799107,2cyclictest7927-21wget12:30:153
1460799107,2cyclictest4826-21taskset09:02:243
1460799107,2cyclictest30428-21taskset11:01:223
1460799107,2cyclictest26265-21wget10:55:153
1460799107,2cyclictest25577-21grep07:30:153
1460799107,2cyclictest25577-21grep07:30:153
1460799107,2cyclictest23605-21tr08:35:143
1460799107,2cyclictest21653-21taskset09:36:523
1460799107,2cyclictest21016-21basename08:30:133
1460799107,2cyclictest20400-21taskset08:25:493
1460799107,2cyclictest18549-21grep11:45:123
1460799107,2cyclictest17054-21chrt07:13:453
1460799107,2cyclictest16650-21uname11:40:193
1460799107,2cyclictest1573-21chrt10:01:063
1460799101,2cyclictest0-21swapper/310:20:133
1460799100,9cyclictest11602-21cut11:30:163
1460299108,1cyclictest30380-21cut12:10:141
1460299108,1cyclictest26750-21taskset07:33:121
1460299108,1cyclictest26750-21taskset07:33:121
1460299108,1cyclictest18899-21fschecks_count11:45:151
1460299108,1cyclictest15010-21taskset09:25:031
1460299108,1cyclictest15010-21taskset09:25:031
1460299107,2cyclictest8445-21grep09:10:171
1460299107,2cyclictest32726-21basename12:15:131
1460299107,2cyclictest32433-21expr07:45:141
1460299107,2cyclictest27727-21tail09:50:161
1460299107,2cyclictest23008-21taskset11:55:021
1460299107,2cyclictest19295-21expr10:40:141
1460299107,2cyclictest12936-21cat10:25:201
1460299107,2cyclictest11659-21grep11:30:171
1460299106,3cyclictest5462-21tr12:25:151
1460299102,2cyclictest225-21systemd-journal08:13:461
1460299100,9cyclictest25379-21wc09:45:171
1460299100,9cyclictest225742sleep111:52:271
1460199108,1cyclictest8794-21chrt08:04:390
1460199108,1cyclictest252212sleep008:36:450
1460199108,1cyclictest22702-21taskset11:54:060
1460199108,1cyclictest2003-21chrt12:16:010
1460199108,1cyclictest193392sleep009:32:380
1460199108,1cyclictest16208-21taskset10:32:580
1460199107,2cyclictest6302-21grep12:25:200
1460199107,2cyclictest30989-21apache_volume11:05:110
1460199107,2cyclictest30534-21date08:49:590
1460199107,2cyclictest30188-21sort09:55:160
1460199107,2cyclictest29660-21chrt12:05:520
1460199107,2cyclictest12570-21cat08:10:200
1460199106,3cyclictest5101-21head07:55:160
1460199106,3cyclictest30529-21tr12:10:150
1460199105,3cyclictest5701-21cat10:10:200
1460199101,1cyclictest0-21swapper/008:05:130
146079996,2cyclictest15201-21egrep09:25:163
146079996,2cyclictest15201-21egrep09:25:163
146079992,3cyclictest225-21systemd-journal08:00:013
146029997,1cyclictest10247-21grep09:15:131
146019997,1cyclictest7993-21cat09:10:140
146019997,1cyclictest43182sleep010:08:080
146019997,1cyclictest23347-21expr11:55:160
146019997,1cyclictest10268-21cat09:15:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional