You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-13 - 15:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Fri Mar 13, 2026 12:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
725594995442,9cyclictest792655-21kworker/u8:0+events_unbound10:35:111
725588995416,24cyclictest0-21swapper/010:33:400
725602995327,22cyclictest786665-21grep10:00:223
725602995227,20cyclictest807324-21grep11:00:243
725602995010,27cyclictest782382-21grep09:50:153
725588995019,13cyclictest736809-21fschecks_time07:40:160
725588995018,13cyclictest802919-21/usr/sbin/munin10:50:110
725594994936,9cyclictest839902-21grep12:35:241
725588994910,23cyclictest833042-21timerwakeupswit12:15:250
725602994812,21cyclictest830233-21latency_hist12:10:013
725594994736,8cyclictest231ktimers/108:15:111
725588994740,4cyclictest744897-21grep08:05:010
725602994627,15cyclictest0-21swapper/311:20:003
725594994635,8cyclictest769388-21grep09:10:241
725588994610,19cyclictest0-21swapper/008:15:240
725588994610,18cyclictest0-21swapper/008:35:000
725602994535,6cyclictest754242-21chrt08:30:143
725602994518,24cyclictest727486-21latency_hist07:15:013
725588994525,11cyclictest0-21swapper/010:40:110
725588994519,9cyclictest807765-21latency_hist11:05:010
725602994326,13cyclictest752122-21cron08:25:003
725602994326,13cyclictest752122-21cron08:25:003
725602994321,18cyclictest778129-21grep09:35:233
72560299431,37cyclictest827381-21/usr/sbin/munin12:00:213
72560299430,37cyclictest0-21swapper/309:25:113
725594994334,5cyclictest744877-21cron08:05:011
725594994330,10cyclictest231ktimers/109:00:121
725588994335,4cyclictest764052-21cron09:00:010
725588994326,13cyclictest729770-21apt-get07:20:090
725588994318,20cyclictest739993-21/usr/sbin/munin07:50:100
725588994317,11cyclictest739907-21grep07:50:010
725602994234,5cyclictest736352-21munin-plugin-st07:40:023
725602994215,24cyclictest835494-21/usr/sbin/munin12:25:183
72560299421,38cyclictest756278-21/usr/sbin/munin08:35:183
72560299421,35cyclictest0-21swapper/309:15:133
725594994231,8cyclictest757245-21sendmail08:40:001
725588994216,9cyclictest0-21swapper/012:25:220
72560299411,35cyclictest818561-21/usr/sbin/munin11:35:193
72560299411,35cyclictest818561-21/usr/sbin/munin11:35:183
72560299410,34cyclictest0-21swapper/307:45:113
725588994134,4cyclictest840367-21latency_hist12:40:000
725588994130,7cyclictest771767-21apt-get09:20:130
725588994129,9cyclictest734873-21/usr/sbin/munin07:35:120
725588994127,11cyclictest826331-21cron11:58:000
725588994125,3cyclictest820696-21sensors_temp11:40:230
725588994125,3cyclictest820696-21sensors_temp11:40:230
725588994118,18cyclictest750870-21fschecks_time08:20:140
725588994118,18cyclictest750870-21fschecks_time08:20:140
725588994114,23cyclictest0-21swapper/009:15:010
725602994028,7cyclictest824807-21debian-sa111:54:593
72560299401,36cyclictest733506-21/usr/sbin/munin07:30:163
725594994030,6cyclictest737648-21unixbench_singl07:40:251
725588994029,7cyclictest811064-21cron11:14:000
725588994028,9cyclictest808268-21/usr/sbin/munin11:05:160
725588994028,9cyclictest770043-21munin-node09:15:120
725588994027,9cyclictest747677-21/usr/sbin/munin08:10:180
725588994027,10cyclictest791448-21/usr/sbin/munin10:15:190
725588994026,10cyclictest816523-21/usr/sbin/munin11:30:120
725588994020,16cyclictest818451-21cut11:35:160
725588994020,16cyclictest818451-21cut11:35:160
72560299392,30cyclictest807992-21apt-get11:05:113
72560299391,35cyclictest763194-21/usr/sbin/munin08:55:223
72560299391,34cyclictest742364-21/usr/sbin/munin07:55:223
72560299391,32cyclictest810327-21/usr/sbin/munin11:10:233
72560299391,32cyclictest805269-21/usr/sbin/munin10:55:233
725602993910,25cyclictest784898-21grep09:55:243
725594993930,4cyclictest0-21swapper/112:20:011
725594993929,5cyclictest727313-21cron07:15:011
725594993928,8cyclictest231ktimers/111:15:111
725588993929,7cyclictest777075-21munin-node09:35:090
725588993927,9cyclictest726265-21/usr/sbin/munin07:10:190
725588993926,10cyclictest773703-21fw_packets09:25:160
725588993925,11cyclictest775487-21df_inode09:30:130
725588993925,10cyclictest794350-21/usr/sbin/munin10:25:100
725588993924,12cyclictest782718-21/usr/sbin/munin09:50:190
725588993920,17cyclictest729464-21sshd07:19:220
725588993916,19cyclictest761259-21latency08:50:190
725602993827,8cyclictest838664-21auditctl12:35:003
72560299382,31cyclictest834129-21/usr/sbin/munin12:20:153
72560299382,31cyclictest791167-21/usr/sbin/munin10:15:153
725602993816,18cyclictest0-21swapper/311:35:003
72560299381,34cyclictest832374-21/usr/sbin/munin12:15:163
72560299381,34cyclictest759631-21/usr/sbin/munin08:45:203
72560299381,32cyclictest801435-21/usr/sbin/munin10:45:163
72560299381,32cyclictest740203-21/usr/sbin/munin07:50:153
72560299381,31cyclictest819938-21apt-get11:40:113
72560299381,31cyclictest819938-21apt-get11:40:113
72560299380,36cyclictest0-21swapper/309:30:113
725597993830,4cyclictest727331-21idleruntime-cro07:15:012
725588993828,7cyclictest801619-21/usr/sbin/munin10:45:190
725588993828,7cyclictest783906-21/usr/sbin/munin09:55:130
725588993828,7cyclictest733289-21diskstats07:30:140
725588993826,9cyclictest815160-21/usr/sbin/munin11:25:180
725588993826,9cyclictest757477-21munin-node08:40:100
725588993826,8cyclictest785939-21/usr/sbin/munin10:00:140
725588993825,10cyclictest787639-21/usr/sbin/munin10:05:140
725588993824,8cyclictest823664-21/usr/sbin/munin11:50:170
725588993823,10cyclictest778933-21/usr/sbin/munin09:40:150
725588993823,10cyclictest752419-21/usr/sbin/munin08:25:150
725588993823,10cyclictest752419-21/usr/sbin/munin08:25:140
725588993820,13cyclictest757420-21latency_hist08:40:010
72560299371,33cyclictest822446-21/usr/sbin/munin11:45:263
72560299371,33cyclictest798428-21/usr/sbin/munin10:35:233
72560299371,33cyclictest767552-21cron09:06:003
72560299371,33cyclictest757997-21/usr/sbin/munin08:40:193
72560299371,33cyclictest746408-21/usr/sbin/munin08:05:263
72560299371,33cyclictest736900-21if_eth107:40:183
72560299371,32cyclictest729873-21df07:20:123
72560299371,31cyclictest830756-21sed12:10:163
72560299371,31cyclictest799577-21/usr/sbin/munin10:40:123
72560299371,31cyclictest731736-21/usr/sbin/munin07:25:173
72560299371,31cyclictest0-21swapper/310:30:113
72560299371,30cyclictest813117-21apt-get11:20:113
72560299370,30cyclictest780504-21apt-get09:45:143
72560299370,30cyclictest780504-21apt-get09:45:133
725597993727,6cyclictest0-21swapper/209:00:012
725597993717,16cyclictest311ktimers/208:10:222
725588993727,7cyclictest834395-21/usr/sbin/munin12:20:200
725588993727,7cyclictest780839-21/usr/sbin/munin09:45:190
725588993727,7cyclictest780839-21/usr/sbin/munin09:45:190
725588993726,8cyclictest837304-21df_inode12:30:130
725588993726,8cyclictest821776-21/usr/sbin/munin11:45:180
725588993726,7cyclictest741997-21/usr/sbin/munin07:55:160
725588993723,11cyclictest829280-21/usr/sbin/munin12:05:210
725588993723,11cyclictest798359-21awk10:35:220
725588993723,11cyclictest0-21swapper/008:05:120
725588993723,10cyclictest759708-21/usr/sbin/munin08:45:220
725588993718,8cyclictest645-21runrttasks11:18:210
72560299361,32cyclictest825663-21/usr/sbin/munin11:55:223
72560299361,32cyclictest792743-21/usr/sbin/munin10:20:123
72560299361,32cyclictest787621-21awk10:05:143
72560299361,32cyclictest779349-21/usr/sbin/munin09:40:203
72560299361,32cyclictest765885-21cron09:04:003
72560299361,32cyclictest749279-21/usr/sbin/munin08:15:173
72560299361,32cyclictest743416-21apt-get08:00:123
72560299361,32cyclictest728853-21/usr/sbin/munin07:15:233
72560299361,31cyclictest839963-21grep12:35:243
72560299361,31cyclictest795039-21/usr/sbin/munin10:25:213
72560299361,30cyclictest752484-21diskstats08:25:173
72560299361,30cyclictest752484-21diskstats08:25:163
72560299361,30cyclictest0-21swapper/309:20:113
725597993628,4cyclictest0-21swapper/207:45:262
72559499363,6cyclictest131rcu_preempt09:30:121
725594993629,4cyclictest7625812sleep108:55:121
725594993628,4cyclictest0-21swapper/109:45:001
725594993626,6cyclictest0-21swapper/111:25:241
725588993627,6cyclictest767094-21/usr/sbin/munin09:05:220
725588993626,7cyclictest766100-21sed09:05:000
725588993626,7cyclictest732855-21sshd07:29:070
725588993624,9cyclictest790261-21/usr/sbin/munin10:10:240
725588993621,11cyclictest827063-21if_eth112:00:160
725588993621,11cyclictest792884-21fschecks_time10:20:140
725588993620,13cyclictest813862-21cut11:20:220
72560299352,30cyclictest802987-21diskstats10:50:133
72560299352,29cyclictest760882-21apt-get08:50:143
72560299351,31cyclictest815056-21/usr/sbin/munin11:25:163
72560299351,31cyclictest747706-21sed08:10:203
725597993525,6cyclictest0-21swapper/210:00:002
725597993523,5cyclictest131rcu_preempt10:55:182
725597993519,9cyclictest131rcu_preempt11:10:142
725597993518,5cyclictest131rcu_preempt07:15:232
725597993513,5cyclictest131rcu_preempt09:20:152
725597993510,8cyclictest32-21ksoftirqd/208:10:002
725594993526,5cyclictest0-21swapper/108:00:021
725594993525,5cyclictest0-21swapper/112:24:561
725588993527,5cyclictest805809-21sshd10:57:200
725588993525,7cyclictest830368-21/usr/sbin/munin12:10:090
72560299341,30cyclictest768460-21diskstats09:10:143
725597993425,6cyclictest776398-21grep09:30:242
725597993424,4cyclictest131rcu_preempt11:00:172
725597993422,4cyclictest131rcu_preempt07:40:162
725597993420,6cyclictest795443-21kworker/u8:2+events_unbound11:23:142
725597993420,3cyclictest131rcu_preempt08:05:012
725597993417,4cyclictest0-21swapper/208:15:162
725597993416,3cyclictest131rcu_preempt08:50:162
725597993416,11cyclictest131rcu_preempt10:00:212
725597993414,12cyclictest131rcu_preempt12:10:232
725597993413,13cyclictest0-21swapper/212:30:172
725594993426,4cyclictest0-21swapper/111:35:001
725597993324,6cyclictest731662-21fschecks_time07:25:162
725597993319,3cyclictest131rcu_preempt12:35:182
725597993317,9cyclictest0-21swapper/212:07:412
725597993315,6cyclictest131rcu_preempt09:15:212
725597993315,5cyclictest131rcu_preempt10:45:172
725597993314,11cyclictest131rcu_preempt12:15:112
72559499333,5cyclictest131rcu_preempt08:20:111
72559499333,5cyclictest131rcu_preempt08:20:101
725594993326,4cyclictest0-21swapper/112:35:001
725597993224,4cyclictest0-21swapper/207:55:152
725597993223,7cyclictest730129-21kernelversion07:20:162
725597993223,4cyclictest131rcu_preempt10:42:382
725597993222,7cyclictest311ktimers/208:25:002
725597993222,7cyclictest311ktimers/208:25:002
725597993221,8cyclictest311ktimers/209:10:002
725597993221,8cyclictest311ktimers/207:55:012
725597993217,9cyclictest131rcu_preempt09:00:112
725597993217,4cyclictest131rcu_preempt08:29:022
725597993217,4cyclictest131rcu_preempt08:29:022
725597993214,5cyclictest131rcu_preempt12:25:142
725597993214,5cyclictest131rcu_preempt10:35:012
725597993214,4cyclictest0-21swapper/209:25:162
725597993214,10cyclictest131rcu_preempt10:10:242
725597993213,5cyclictest131rcu_preempt11:45:262
725597993213,5cyclictest131rcu_preempt07:30:212
725597993212,4cyclictest0-21swapper/210:40:012
725597993211,5cyclictest131rcu_preempt09:45:242
725597993211,5cyclictest131rcu_preempt09:45:232
725594993224,4cyclictest0-21swapper/108:10:241
72559799318,19cyclictest311ktimers/212:05:012
725597993123,5cyclictest0-21swapper/208:45:162
725597993120,7cyclictest311ktimers/209:35:192
725597993119,4cyclictest131rcu_preempt10:23:372
725597993115,9cyclictest131rcu_preempt12:20:152
725597993115,5cyclictest131rcu_preempt09:15:012
725597993114,5cyclictest131rcu_preempt08:35:172
725597993112,4cyclictest131rcu_preempt11:40:182
725597993112,4cyclictest131rcu_preempt11:40:182
725597993110,3cyclictest0-21swapper/211:30:132
725594993124,4cyclictest0-21swapper/108:30:231
725594993122,5cyclictest208-21systemd-logind11:49:491
72559799302,15cyclictest32-21ksoftirqd/211:55:122
725597993019,8cyclictest311ktimers/210:10:012
725597993014,5cyclictest131rcu_preempt10:15:232
725597993014,5cyclictest131rcu_preempt09:50:192
725597993013,5cyclictest131rcu_preempt11:38:552
725597993013,5cyclictest131rcu_preempt11:38:552
725597993012,10cyclictest0-21swapper/211:05:192
725597993011,4cyclictest0-21swapper/211:16:012
725597993011,11cyclictest131rcu_preempt10:25:102
725594993022,4cyclictest0-21swapper/109:20:151
725594993019,6cyclictest785225-21kworker/u8:1+events_unbound10:15:241
72559799299,4cyclictest0-21swapper/208:41:022
72559799297,5cyclictest131rcu_preempt10:50:222
725597992918,4cyclictest131rcu_preempt08:30:132
725597992913,4cyclictest131rcu_preempt11:54:102
725597992910,11cyclictest32-21ksoftirqd/211:25:192
72559499296,15cyclictest709351-21kworker/u8:2+events_unbound07:35:161
725594992921,4cyclictest0-21swapper/108:25:261
725594992921,4cyclictest0-21swapper/108:25:261
725594992921,3cyclictest0-21swapper/109:55:251
725594992920,4cyclictest0-21swapper/110:55:151
725597992814,3cyclictest131rcu_preempt09:40:132
725597992810,10cyclictest131rcu_preempt07:35:132
725594992820,6cyclictest827841-21kworker/u8:2+events_unbound12:08:401
725594992720,4cyclictest0-21swapper/110:05:241
725594992720,4cyclictest0-21swapper/109:15:251
725594992720,4cyclictest0-21swapper/108:40:251
725594992720,4cyclictest0-21swapper/107:25:271
725594992719,5cyclictest755153-21kworker/u8:3+events_unbound08:50:221
72559499269,3cyclictest0-21swapper/108:49:561
725594992621,3cyclictest725583-21cyclictest11:00:101
725594992620,3cyclictest0-21swapper/109:30:001
725594992619,4cyclictest0-21swapper/111:59:591
72559499261,8cyclictest0-21swapper/110:00:091
725594992618,5cyclictest822774-21kworker/u8:0+events_unbound12:14:221
725594992618,5cyclictest0-21swapper/111:50:241
725594992615,7cyclictest809003-21grep11:05:241
72559499261,4cyclictest131rcu_preempt11:35:121
72559499261,4cyclictest131rcu_preempt11:35:121
725594992614,9cyclictest725583-21cyclictest10:20:101
725594992611,5cyclictest131rcu_preempt10:45:201
725594992519,4cyclictest4179903-21kworker/u8:0+events_unbound07:46:501
72559499251,20cyclictest781544-21grep09:45:271
72559499251,20cyclictest781544-21grep09:45:271
725594992418,4cyclictest739711-21kworker/u8:2+events_unbound07:50:161
725594992417,4cyclictest821403-21sh11:45:011
725594992417,4cyclictest821403-21sh11:45:001
725594992416,6cyclictest231ktimers/107:20:171
72559499241,3cyclictest799471-21apt-get10:40:111
725594992317,3cyclictest0-21swapper/107:15:221
72559499231,5cyclictest803905-21unixbench_singl10:50:241
72559499228,3cyclictest131rcu_preempt07:33:131
72559499222,5cyclictest131rcu_preempt10:25:121
725594992216,3cyclictest0-21swapper/108:05:201
725594992215,4cyclictest797476-21sshd10:34:361
725594992214,4cyclictest0-21swapper/109:09:241
725594992213,5cyclictest231ktimers/111:25:011
725594992114,5cyclictest836612-21kworker/u8:1+events_unbound12:27:201
725594992113,5cyclictest231ktimers/109:50:221
725594992113,4cyclictest438-21cron12:05:001
72559499211,15cyclictest789763-21ntpq10:10:191
725594992012,6cyclictest810196-21missed_timers11:10:201
725594991811,5cyclictest231ktimers/109:35:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional