You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-12 - 22:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Wed Aug 12, 2026 12:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1508399521,20cyclictest26580-21ssh10:00:131
1509499445,17cyclictest21528-21ssh11:45:142
1509499437,11cyclictest27-21ksoftirqd/211:15:132
15094994319,18cyclictest27-21ksoftirqd/208:40:152
1508399423,24cyclictest28051-21ssh09:25:121
1507499411,31cyclictest25295-21ssh12:05:170
1507499404,30cyclictest19796-21sh12:20:150
1509499392,19cyclictest30347-21sh10:20:152
1509499371,20cyclictest6902-21ssh10:25:142
1510399362,20cyclictest35966irq/58-eth011:25:123
1509499363,20cyclictest27-21ksoftirqd/210:05:152
1509499363,20cyclictest27-21ksoftirqd/210:05:142
1510399351,23cyclictest35966irq/58-eth011:50:143
1509499353,9cyclictest0-21swapper/211:28:282
15094993513,16cyclictest15056-21cyclictest11:05:122
1510399343,21cyclictest35966irq/58-eth010:05:143
1510399343,21cyclictest35966irq/58-eth010:05:143
1507499342,25cyclictest15926-21ssh10:30:130
15103993317,12cyclictest0-21swapper/311:40:153
15103993215,12cyclictest0-21swapper/309:15:133
1508399321,23cyclictest8868-21ssh11:20:141
15103993116,10cyclictest35966irq/58-eth012:30:153
1508399312,24cyclictest31883-21ssh09:45:141
1510399304,16cyclictest35966irq/58-eth010:50:143
15103993017,9cyclictest35966irq/58-eth012:05:163
15103993015,11cyclictest0-21swapper/310:25:143
15103993011,13cyclictest35966irq/58-eth012:25:133
1509499301,22cyclictest13800-21ssh09:35:152
1508399303,24cyclictest32221-21ssh11:15:121
15074993022,4cyclictest0-21swapper/009:20:130
1509499292,21cyclictest6435-21ssh11:54:462
1509499292,21cyclictest0-21swapper/208:45:132
1509499292,17cyclictest27346irq/40-dwc2_hso07:25:152
1509499292,17cyclictest27346irq/40-dwc2_hso07:25:142
1509499292,11cyclictest24244-21ssh09:57:462
1509499291,19cyclictest1740-21sh12:10:122
15094992910,14cyclictest27-21ksoftirqd/208:55:122
15083992912,12cyclictest730-21diskmemload11:58:471
1507499292,23cyclictest21322-21ssh10:15:140
1507499292,22cyclictest14656-21sh11:58:460
1510399283,13cyclictest35966irq/58-eth010:45:133
1510399282,15cyclictest35966irq/58-eth008:00:143
1509499283,16cyclictest27346irq/40-dwc2_hso08:30:132
1509499283,14cyclictest27346irq/40-dwc2_hso11:00:142
1509499281,18cyclictest23157-21ssh11:10:132
15094992710,4cyclictest27-21ksoftirqd/210:00:342
1508399272,18cyclictest1152-21ssh09:10:131
15083992713,4cyclictest21-21ksoftirqd/107:45:021
1508399271,21cyclictest19797-21ssh12:20:141
1507499272,18cyclictest1273-21ssh10:03:090
15074992712,3cyclictest131rcu_preempt11:30:250
1510399264,12cyclictest35966irq/58-eth010:30:133
15103992615,8cyclictest0-21swapper/309:35:133
1509499267,8cyclictest261rcuc/211:30:132
1509499262,19cyclictest21968-21ssh12:38:462
1509499262,19cyclictest21968-21ssh12:38:452
1509499261,18cyclictest4613-21ssh09:30:052
1508399262,18cyclictest19003-21ssh10:12:441
15083992613,7cyclictest0-21swapper/112:30:131
1508399260,20cyclictest0-21swapper/108:25:141
1510399252,16cyclictest35966irq/58-eth011:00:143
15103992519,3cyclictest0-21swapper/309:25:133
15103992513,8cyclictest35966irq/58-eth012:10:143
1510399251,13cyclictest35966irq/58-eth009:20:143
1509499253,14cyclictest27346irq/40-dwc2_hso12:15:142
1509499252,16cyclictest0-21swapper/210:45:442
1509499252,15cyclictest27346irq/40-dwc2_hso08:05:132
15094992514,8cyclictest27-21ksoftirqd/209:46:202
1507499256,13cyclictest0-21swapper/012:00:140
1507499253,16cyclictest12500-21ssh11:40:140
15074992513,7cyclictest0-21swapper/010:10:450
15074992512,11cyclictest12-21ksoftirqd/009:40:110
1510399243,14cyclictest0-21swapper/310:03:093
1510399242,12cyclictest0-21swapper/311:45:133
15103992419,2cyclictest0-21swapper/307:30:133
1509499245,13cyclictest27-21ksoftirqd/210:35:132
1509499243,15cyclictest27346irq/40-dwc2_hso07:30:122
1509499243,13cyclictest27346irq/40-dwc2_hso07:55:112
15094992412,10cyclictest4817-21cron11:00:012
1508399241,16cyclictest22864-21ssh09:40:121
1507499248,3cyclictest131rcu_preempt07:50:000
15074992411,9cyclictest30353-21diskmemload10:20:160
1507499241,18cyclictest1151-21sh09:10:130
1510399235,10cyclictest0-21swapper/308:20:143
1510399233,14cyclictest0-21swapper/310:12:453
15103992314,3cyclictest0-21swapper/311:10:123
15103992311,9cyclictest35966irq/58-eth011:20:123
1509499232,6cyclictest0-21swapper/212:04:452
1509499232,6cyclictest0-21swapper/210:14:462
1509499232,16cyclictest27346irq/40-dwc2_hso07:50:122
1509499232,15cyclictest0-21swapper/208:15:042
1509499231,11cyclictest16495-21irqrtprio10:30:182
1508399231,19cyclictest32376-21ls08:35:171
1508399231,18cyclictest19118-21ssh09:20:151
1508399231,16cyclictest730-21diskmemload12:25:131
1508399231,16cyclictest10122-21ssh09:15:141
1510399229,8cyclictest0-21swapper/311:30:143
1510399229,10cyclictest0-21swapper/309:30:123
1510399223,13cyclictest35966irq/58-eth010:40:123
1509499229,10cyclictest19939-21if_err_eth110:50:182
1509499228,9cyclictest0-21swapper/208:35:162
1509499228,11cyclictest730-21diskmemload11:20:292
1509499227,6cyclictest27-21ksoftirqd/210:15:132
1509499222,8cyclictest6904-21fschecks_count07:35:152
1509499222,16cyclictest0-21swapper/210:44:452
1509499222,15cyclictest0-21swapper/211:42:452
1509499221,17cyclictest1205-21kthreadcore08:00:232
1509499221,13cyclictest20846-21rm09:20:222
15094992210,9cyclictest9280-21ssh11:36:402
1508399224,15cyclictest8494-21ssh09:50:131
1508399222,16cyclictest16170-21ssh12:00:161
15083992211,4cyclictest730-21diskmemload11:27:431
1508399220,17cyclictest29861-21ls07:25:181
1508399220,17cyclictest29861-21ls07:25:181
15074992210,4cyclictest131rcu_preempt10:05:300
15074992210,4cyclictest131rcu_preempt10:05:290
1510399218,8cyclictest35966irq/58-eth011:55:113
1510399218,10cyclictest0-21swapper/309:45:203
15103992114,4cyclictest0-21swapper/308:35:123
15103992114,3cyclictest0-21swapper/311:35:133
1510399211,16cyclictest35966irq/58-eth008:15:123
1509499217,11cyclictest4924-21latency_hist12:30:012
1509499212,7cyclictest0-21swapper/208:10:132
1509499212,16cyclictest8028-21kthreadcore12:30:232
1509499211,9cyclictest1573-21sed09:10:182
15094992112,7cyclictest0-21swapper/209:43:572
1509499211,17cyclictest23232-21latency12:20:252
1509499211,10cyclictest8499-21ssh09:50:142
1509499211,10cyclictest24972-21fschecks_time07:20:172
15094992110,9cyclictest0-21swapper/211:56:162
1509499210,12cyclictest13803-21ps09:15:272
1508399215,13cyclictest21-21ksoftirqd/110:30:111
1508399211,6cyclictest31838-21ssh11:15:011
1508399211,3cyclictest0-21swapper/110:57:051
15083992111,7cyclictest27588-21grep09:00:271
15083992111,3cyclictest730-21diskmemload12:37:381
15083992111,3cyclictest730-21diskmemload12:37:371
15083992111,3cyclictest730-21diskmemload11:30:521
1507499215,11cyclictest0-21swapper/011:15:120
15074992112,7cyclictest3755-21sshd10:58:460
15074992110,8cyclictest4023-21cat09:45:320
1510399208,9cyclictest35966irq/58-eth009:10:133
1510399205,11cyclictest0-21swapper/310:15:173
15103992014,2cyclictest0-21swapper/307:45:123
15103992012,3cyclictest0-21swapper/312:35:123
15103992012,3cyclictest0-21swapper/312:35:113
15103992010,7cyclictest0-21swapper/310:35:123
1509499207,10cyclictest27073-21cron08:30:002
1509499207,10cyclictest18040-21irqcore08:20:192
1509499202,9cyclictest32443-21grep09:05:272
1509499202,9cyclictest18256-21kthreadcore07:10:232
1509499202,15cyclictest261rcuc/208:50:132
1509499201,5cyclictest0-21swapper/207:45:152
1509499201,15cyclictest26830-21ssh12:05:252
1509499201,10cyclictest32102-21timerwakeupswit09:25:282
1509499201,10cyclictest27327-21grep09:00:262
1509499201,10cyclictest14839-21kthreadcore07:40:212
1508399201,8cyclictest6495-21munin-run10:25:011
1508399201,2cyclictest0-21swapper/112:08:491
1508399201,2cyclictest0-21swapper/111:38:251
1508399201,10cyclictest0-21swapper/111:08:011
15083992010,3cyclictest730-21diskmemload10:50:481
1508399200,15cyclictest18826-21ls07:10:271
1507499203,12cyclictest14319-21ssh12:35:140
1507499203,12cyclictest14319-21ssh12:35:130
1507499201,15cyclictest10121-21sh09:15:140
1507499201,14cyclictest14921-21missed_timers07:40:230
1510399199,6cyclictest0-21swapper/309:54:563
15103991914,1cyclictest0-21swapper/309:55:123
15103991913,3cyclictest0-21swapper/307:50:143
15103991912,3cyclictest0-21swapper/308:45:123
1510399191,12cyclictest0-21swapper/309:05:273
1510399191,11cyclictest0-21swapper/307:10:233
15103991910,7cyclictest0-21swapper/311:15:123
15103991910,6cyclictest0-21swapper/309:00:103
1509499191,9cyclictest22258-21kthreadcore07:15:202
1508399199,7cyclictest0-21swapper/109:55:261
1508399192,3cyclictest131rcu_preempt11:45:211
1508399192,13cyclictest4293-21apt-get09:30:031
1508399192,12cyclictest24987-21ssh10:35:141
1508399191,5cyclictest6885-21apt-get10:25:151
1508399191,3cyclictest0-21swapper/112:18:071
15083991912,5cyclictest9174-21sshd12:13:381
15083991911,6cyclictest1-21systemd07:57:491
1508399191,14cyclictest10582-21ssh10:45:151
1508399190,15cyclictest13077-21timerwakeupswit08:45:281
1507499199,8cyclictest14350-21fw_packets08:50:170
1507499199,7cyclictest24814-21apt-get07:20:130
1507499199,7cyclictest19048-21df_inode08:55:140
1507499198,8cyclictest10557-21ssh10:45:130
1507499197,9cyclictest734-21sh09:10:020
1507499196,7cyclictest0-21swapper/012:25:140
1507499193,13cyclictest21518-21apt-get11:45:130
15074991913,2cyclictest10725-21ssh12:15:120
1507499191,16cyclictest6682-21ls11:55:020
1507499191,14cyclictest7210-21rm11:35:290
1510399189,6cyclictest0-21swapper/312:23:563
1510399188,8cyclictest0-21swapper/312:16:113
1510399183,10cyclictest0-21swapper/310:59:463
1510399182,8cyclictest0-21swapper/308:50:123
1510399182,10cyclictest0-21swapper/307:20:143
1510399181,11cyclictest35966irq/58-eth009:40:113
1510399181,10cyclictest35966irq/58-eth008:40:143
15103991810,5cyclictest0-21swapper/310:20:143
1508399186,7cyclictest21-21ksoftirqd/108:05:141
1508399183,11cyclictest0-21swapper/107:45:141
1508399182,11cyclictest0-21swapper/110:15:151
1508399181,9cyclictest20064-21tr08:55:211
1508399181,3cyclictest0-21swapper/111:02:111
1508399181,13cyclictest1547-21sh10:40:131
15083991810,5cyclictest2227-21usage_Anybus11:50:311
1507499188,8cyclictest17712-21apt-get08:20:150
1507499188,7cyclictest18005-21fschecks_time11:25:160
1507499186,9cyclictest29850-21grep07:55:260
1507499186,8cyclictest12-21ksoftirqd/009:01:460
1507499184,10cyclictest664-21ntpd09:50:110
1507499182,13cyclictest17916-21tr09:55:180
1507499181,3cyclictest131rcu_preempt09:35:210
15074991811,4cyclictest0-21swapper/007:50:270
1507499181,13cyclictest9602-21kthreadcore11:20:200
1507499181,13cyclictest9209-21ls08:45:020
1507499181,13cyclictest5433-21diskstats12:30:160
1507499181,11cyclictest18069-21ls11:05:280
1507499181,10cyclictest8229-21perl08:10:160
15074991810,6cyclictest18607-21nfs_client07:10:240
1510399179,6cyclictest0-21swapper/312:00:253
1510399179,6cyclictest0-21swapper/311:07:303
15103991711,3cyclictest0-21swapper/308:30:113
1508399179,6cyclictest22243-21sshd09:39:451
1508399175,9cyclictest23010-21kthreadcore07:15:211
1508399171,9cyclictest28573-21/usr/sbin/munin09:05:281
1508399171,2cyclictest131rcu_preempt10:06:211
1508399171,2cyclictest131rcu_preempt10:06:201
1508399171,13cyclictest2185-21fschecks_time07:30:171
1508399171,12cyclictest12501-21diskmemload11:40:151
15083991710,4cyclictest8849-21cut08:10:181
1507499179,6cyclictest10123-21sshd12:14:470
1507499178,7cyclictest31687-21nfs_client09:25:240
1507499178,6cyclictest8384-21cut10:25:220
1507499177,8cyclictest5460-21ntp_states07:30:260
1507499177,7cyclictest8392-21kthreadcore11:00:240
1507499176,3cyclictest131rcu_preempt10:40:180
1507499174,10cyclictest8972-21cat09:30:300
1507499172,9cyclictest0-21swapper/010:35:110
1507499172,12cyclictest6796-21apt-get07:35:120
1507499172,10cyclictest0-21swapper/008:25:150
1507499171,8cyclictest5273-21kthreadcore08:05:210
15074991711,4cyclictest386-21latency07:25:240
15074991711,4cyclictest386-21latency07:25:230
15074991711,4cyclictest19611-21sh10:50:140
1507499171,13cyclictest27310-21awk11:10:290
1510399161,7cyclictest0-21swapper/308:10:013
15103991611,3cyclictest0-21swapper/308:10:133
15103991611,2cyclictest0-21swapper/307:55:103
15103991611,2cyclictest0-21swapper/307:25:133
15103991611,2cyclictest0-21swapper/307:25:123
1509499160,7cyclictest15134-21latency_hist07:10:002
1508399168,6cyclictest27800-21sed08:30:171
1508399162,11cyclictest15605-21kthreadcore08:15:221
1508399161,8cyclictest9812-21grep07:35:201
15083991610,4cyclictest0-21swapper/108:00:301
1508399160,13cyclictest26680-21kthreadcore07:20:221
1508399160,11cyclictest8236-21timerandwakeup08:40:291
1507499161,5cyclictest17489-21grep08:20:010
15074991610,4cyclictest3206-21proc_pri08:35:270
15074991610,4cyclictest0-21swapper/008:00:160
1510399155,7cyclictest0-21swapper/307:40:293
1510399152,9cyclictest0-21swapper/307:15:123
15103991511,2cyclictest0-21swapper/308:55:123
1508399152,9cyclictest0-21swapper/108:50:141
1508399151,10cyclictest21351-21grep07:50:171
1508399150,12cyclictest17942-21ls08:20:171
1507499152,10cyclictest9758-21irqcore08:45:180
1507499151,11cyclictest27310-21apt-get08:30:130
1507499151,11cyclictest22604-21kthreadcore07:15:210
1510399149,3cyclictest0-21swapper/308:25:133
15103991410,2cyclictest0-21swapper/307:35:113
1508399141,10cyclictest15133-21latency_hist07:10:011
1507499131,9cyclictest15076-21latency_hist07:10:000
151039994,3cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional