You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-17 - 22:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Mon Aug 17, 2026 12:45:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1247995435,13cyclictest35966irq/58-eth011:45:133
124799454,26cyclictest35966irq/58-eth011:15:123
122699411,4cyclictest131rcu_preempt09:25:141
122699411,4cyclictest131rcu_preempt09:25:141
124799404,24cyclictest35966irq/58-eth011:30:133
1247993920,11cyclictest0-21swapper/308:40:143
122699381,23cyclictest14663-21ssh12:25:141
124799375,22cyclictest35966irq/58-eth012:25:143
124799363,18cyclictest35966irq/58-eth011:50:123
124799362,19cyclictest0-21swapper/311:05:133
124799352,20cyclictest35966irq/58-eth009:40:173
124799344,19cyclictest0-21swapper/309:30:123
1247993418,9cyclictest35966irq/58-eth012:15:143
121499340,23cyclictest14359-21ssh09:25:130
121499340,23cyclictest14359-21ssh09:25:130
124799322,16cyclictest35966irq/58-eth009:25:133
124799322,16cyclictest35966irq/58-eth009:25:133
1247993017,6cyclictest0-21swapper/309:15:123
122699309,17cyclictest21-21ksoftirqd/111:05:141
1214992911,2cyclictest12-21ksoftirqd/010:00:130
124799284,14cyclictest35966irq/58-eth009:00:123
124799283,18cyclictest0-21swapper/312:24:533
124799282,15cyclictest35966irq/58-eth008:45:143
122699281,20cyclictest23635-21ssh12:30:131
124799273,15cyclictest0-21swapper/312:11:053
121499271,3cyclictest131rcu_preempt12:25:150
124799264,15cyclictest0-21swapper/311:25:153
124799253,17cyclictest0-21swapper/312:06:043
1247992512,9cyclictest35966irq/58-eth010:35:143
1247992511,8cyclictest0-21swapper/310:45:143
123599251,17cyclictest4037-21ssh11:25:152
122699251,18cyclictest29087-21ssh12:15:141
124799248,10cyclictest0-21swapper/311:02:533
124799242,14cyclictest0-21swapper/309:10:153
1247992411,7cyclictest35966irq/58-eth010:10:163
124799240,20cyclictest0-21swapper/307:35:123
123599241,10cyclictest16131-21idleruntime-cro11:50:012
124799238,9cyclictest35966irq/58-eth011:35:133
124799238,9cyclictest35966irq/58-eth011:35:133
1247992317,2cyclictest0-21swapper/308:35:143
123599239,11cyclictest28240-21cron09:15:012
123599234,10cyclictest32213-21munin-run12:35:012
123599232,13cyclictest5706-21ntp_192.168.11412:00:252
123599232,13cyclictest5706-21ntp_192.168.11412:00:252
122699236,11cyclictest21-21ksoftirqd/109:10:131
121499238,3cyclictest131rcu_preempt09:15:260
121499231,17cyclictest27483-21ssh11:20:120
121499231,17cyclictest2099-21ssh10:30:160
124799223,12cyclictest0-21swapper/311:40:143
1247992215,5cyclictest35966irq/58-eth011:10:153
1247992212,4cyclictest0-21swapper/310:25:133
1247992210,7cyclictest35966irq/58-eth011:55:123
123599229,10cyclictest27928-21ssh12:14:072
123599228,11cyclictest30273-21df_inode07:40:162
123599227,12cyclictest16302-21cut09:25:212
123599227,12cyclictest16302-21cut09:25:212
123599221,6cyclictest31671-21irqstats11:40:182
1235992211,9cyclictest23119-21ssh11:52:392
123599221,13cyclictest9014-21ssh09:40:172
123599221,12cyclictest30763-21ssh10:10:172
123599221,12cyclictest14796-21ssh10:55:132
123599221,12cyclictest12943-21ssh11:30:132
123599221,11cyclictest1586-21apt-get10:30:022
1235992210,4cyclictest27-21ksoftirqd/211:05:142
1235992210,4cyclictest23241-21latency_hist12:30:012
1226992214,5cyclictest0-21swapper/110:55:261
122699221,13cyclictest32396-21rm09:35:141
122699221,10cyclictest1309166fschecks_count11:30:151
121499227,4cyclictest19209-21diskmemload10:15:130
121499222,7cyclictest16620-21df11:50:150
121499221,6cyclictest91-21pr/ttyS011:15:010
1214992210,9cyclictest12740-21tr10:35:210
121499220,16cyclictest12446-21ssh11:29:530
124799219,4cyclictest35966irq/58-eth010:20:143
124799213,12cyclictest0-21swapper/312:02:533
124799213,12cyclictest0-21swapper/312:02:533
1247992113,6cyclictest0-21swapper/310:00:263
123599219,9cyclictest0-21swapper/210:20:042
123599219,5cyclictest10722-21sshd07:18:532
123599213,14cyclictest844-21munin-node07:55:122
123599212,12cyclictest29185-21ssh10:45:142
123599212,11cyclictest25452-21ssh11:55:122
1235992116,2cyclictest27-21ksoftirqd/211:00:122
1235992115,3cyclictest11080-21sh12:05:152
1235992114,4cyclictest0-21swapper/209:00:282
123599211,11cyclictest29067-21ssh12:15:132
122699217,4cyclictest19209-21diskmemload12:24:331
122699215,3cyclictest131rcu_preempt12:05:011
122699215,3cyclictest131rcu_preempt12:05:011
122699211,4cyclictest0-21swapper/110:15:191
122699211,17cyclictest22792-21awk08:40:011
121499219,3cyclictest12-21ksoftirqd/012:33:540
121499216,3cyclictest19209-21diskmemload11:47:000
121499213,2cyclictest131rcu_preempt11:39:230
121499213,2cyclictest131rcu_preempt11:39:230
121499213,16cyclictest21490-21sh08:05:010
121499211,7cyclictest28686-21munin-run12:15:010
1214992114,4cyclictest14789-21sh10:55:120
1214992110,8cyclictest31313-21apt-get08:15:130
1214992110,3cyclictest12-21ksoftirqd/009:24:520
121499210,16cyclictest27336-21ls11:00:220
124799207,9cyclictest35966irq/58-eth011:20:123
124799207,10cyclictest33-21ksoftirqd/310:40:273
124799207,10cyclictest33-21ksoftirqd/310:40:273
124799202,12cyclictest35966irq/58-eth010:30:163
1247992013,4cyclictest0-21swapper/310:15:293
123599209,4cyclictest23872-21irqcore09:30:182
123599207,5cyclictest20625-21ssh10:40:202
123599207,5cyclictest20625-21ssh10:40:202
123599207,10cyclictest10318-21ssh10:34:322
123599207,10cyclictest0-21swapper/207:50:292
123599206,6cyclictest5337-21kthreadcore08:20:212
123599205,11cyclictest27466-21apt-get11:20:112
123599203,8cyclictest25482-21tune2fs07:35:152
123599202,16cyclictest21968-21rm11:35:132
123599202,16cyclictest21968-21rm11:35:132
123599202,15cyclictest8269-21usage_EthcTask09:55:332
123599202,10cyclictest21950-21grep08:05:182
123599202,10cyclictest14253-21wc07:20:232
123599201,7cyclictest607566ssh09:37:032
123599201,7cyclictest5124-21ls09:20:022
1235992013,5cyclictest9692-21/usr/sbin/munin11:10:182
123599201,11cyclictest6068-21irqcore12:20:182
123599201,11cyclictest20433-21grep08:00:272
123599201,11cyclictest16342-21ntpq10:00:272
123599201,10cyclictest854-21sshd08:35:252
123599201,10cyclictest5928-21idleruntime-cro07:15:012
123599201,10cyclictest3441-21users08:15:322
123599201,10cyclictest323-21diskstats12:35:152
123599201,10cyclictest22468-21ps10:05:212
123599201,10cyclictest21895-21ssh09:45:282
123599201,10cyclictest12525-21ssh10:35:202
122699209,3cyclictest21-21ksoftirqd/110:40:171
122699209,3cyclictest21-21ksoftirqd/110:40:171
122699206,3cyclictest131rcu_preempt10:25:281
122699202,4cyclictest12225-21munin-run10:00:011
122699202,11cyclictest7266-21sendmail_mailqu08:20:281
122699201,8cyclictest0-21swapper/112:05:131
122699201,16cyclictest26726-21ssh09:50:021
122699201,14cyclictest11125-21ssh10:35:141
121499204,3cyclictest19209-21diskmemload12:01:460
121499204,3cyclictest19209-21diskmemload12:01:460
121499204,3cyclictest19209-21diskmemload10:51:470
121499203,15cyclictest6266-21fschecks_count07:15:160
121499203,15cyclictest3346-21kthreadcore09:35:230
121499202,7cyclictest0-21swapper/010:29:320
121499201,8cyclictest0-21swapper/009:14:050
121499201,14cyclictest24075-21ls10:40:310
121499201,14cyclictest24075-21ls10:40:310
1214992010,7cyclictest2250-21ntpq11:40:260
124799199,7cyclictest0-21swapper/312:35:193
124799196,6cyclictest0-21swapper/309:45:123
124799193,11cyclictest0-21swapper/310:55:503
124799193,11cyclictest0-21swapper/310:54:533
124799192,11cyclictest0-21swapper/308:10:133
124799192,11cyclictest0-21swapper/308:10:133
124799191,13cyclictest0-21swapper/309:05:263
124799191,12cyclictest0-21swapper/308:25:173
123599196,5cyclictest21019-21cut07:30:182
123599193,8cyclictest16648-21tr07:25:192
123599192,8cyclictest7602-21grep10:15:162
123599192,8cyclictest27003-21irqrtprio08:10:182
123599192,8cyclictest27003-21irqrtprio08:10:182
123599191,9cyclictest924-21kthreadcore08:50:192
123599191,9cyclictest924-21kthreadcore08:50:192
123599191,9cyclictest8295-21kthreadcore10:50:222
123599191,9cyclictest5298-21/usr/sbin/munin09:20:172
123599191,9cyclictest27190-21cut09:50:142
123599191,9cyclictest14630-21apt-get09:05:132
123599191,9cyclictest11923-21grep08:25:262
123599191,10cyclictest2637-21diskstats07:45:162
123599191,10cyclictest23122-21diskstats08:40:172
123599191,10cyclictest18556-21df_inode11:15:142
123599191,10cyclictest16695-21grep08:30:222
122699193,3cyclictest131rcu_preempt11:20:211
122699192,14cyclictest30378-21kthreadcore09:50:211
122699192,11cyclictest26120-21ssh11:18:531
122699191,8cyclictest0-21swapper/111:47:181
122699191,7cyclictest131rcu_preempt09:42:041
122699191,7cyclictest0-21swapper/110:05:441
122699191,3cyclictest131rcu_preempt11:30:001
1226991912,5cyclictest9489-21ssh10:33:311
122699191,16cyclictest29633-21kthreadcore08:10:211
122699191,16cyclictest29633-21kthreadcore08:10:211
1226991911,6cyclictest0-21swapper/110:45:231
122699191,15cyclictest3295-21kthreadcore07:45:191
122699191,15cyclictest25075-21latency_hist10:25:011
122699191,15cyclictest23016-21kthreadcore08:05:211
122699191,15cyclictest20736-21grep08:00:291
122699191,13cyclictest12655-21ssh10:00:151
122699191,12cyclictest3713-21ssh11:40:531
121499199,2cyclictest12-21ksoftirqd/009:42:040
121499194,12cyclictest26575-21ssh10:05:330
121499193,14cyclictest21326-21sshd10:20:380
121499193,14cyclictest13928-21tr07:55:190
121499193,13cyclictest9592-21sh09:00:010
121499193,13cyclictest1446-21tr10:10:250
121499191,6cyclictest13566-21timerwakeupswit09:00:270
121499191,4cyclictest0-21swapper/009:55:330
1214991913,3cyclictest424-21cpuspeed11:05:150
1214991912,5cyclictest0-21swapper/007:20:180
1214991911,6cyclictest29339-21sshd09:31:530
1214991911,6cyclictest1-21systemd12:21:060
121499191,12cyclictest25293-21ssh11:17:530
1214991910,6cyclictest7191-21sshd12:37:570
121499190,14cyclictest22980-21ls08:40:020
124799189,6cyclictest0-21swapper/309:55:323
124799189,6cyclictest0-21swapper/309:50:153
124799187,9cyclictest0-21swapper/312:32:463
124799182,11cyclictest36066irq/59-eth009:20:123
124799182,10cyclictest0-21swapper/307:10:143
123599181,9cyclictest5430-21irqcore08:55:172
123599181,9cyclictest27874-21cpu08:45:152
123599181,13cyclictest1288-21latency_hist07:10:022
122699182,13cyclictest31149-21tr09:15:211
122699182,13cyclictest1437-21kthreadcore10:10:251
122699182,12cyclictest25138-21kthreadcore09:30:211
122699181,8cyclictest23259-21kthreadcore11:35:191
122699181,8cyclictest23259-21kthreadcore11:35:191
122699181,7cyclictest8459-21tr07:15:211
1226991813,3cyclictest5332-21ssh09:20:131
122699181,15cyclictest1412-21kthreadcore12:35:191
122699181,14cyclictest21311-21kthreadcore07:30:191
122699181,14cyclictest18044-21cat08:35:021
122699181,14cyclictest14395-21cron09:05:011
122699181,14cyclictest1320-21cat07:10:021
122699180,1cyclictest0-21swapper/110:52:341
121499188,7cyclictest30267-21df07:40:150
121499183,13cyclictest29350-21kthreadcore08:45:220
121499183,13cyclictest13916-21sed08:30:170
121499183,12cyclictest23708-21ps08:40:200
121499182,4cyclictest12-21ksoftirqd/010:45:200
121499182,13cyclictest29783-21kthreadcore09:50:200
121499181,3cyclictest131rcu_preempt09:45:320
1214991812,4cyclictest0-21swapper/008:10:240
1214991812,4cyclictest0-21swapper/008:10:240
1214991811,5cyclictest2023-21ssh12:16:050
1214991811,4cyclictest26132-21cut11:55:170
121499181,10cyclictest16551-21ntp_192.168.11411:30:230
124799179,6cyclictest0-21swapper/309:35:113
124799172,9cyclictest0-21swapper/307:55:123
124799172,11cyclictest0-21swapper/310:05:213
124799171,12cyclictest0-21swapper/308:05:133
122699174,3cyclictest19209-21diskmemload12:14:401
122699171,14cyclictest28186-21cut07:35:211
122699171,13cyclictest30872-21cut07:40:201
122699171,13cyclictest17358-21cut09:05:201
122699171,13cyclictest12117-21apt-get07:55:111
122699171,11cyclictest27326-21ssh11:00:221
122699171,10cyclictest365-21ssh11:58:331
122699171,10cyclictest21381-21ssh11:50:341
121499179,6cyclictest3930-21cut07:45:200
121499179,6cyclictest19404-21cat09:10:020
121499173,12cyclictest5928-21cut08:20:220
121499173,12cyclictest17483-21cut07:25:200
121499172,13cyclictest1481-21fschecks_count07:10:160
121499172,13cyclictest10712-21kthreadcore08:25:220
121499171,9cyclictest14808-21ssh12:05:280
121499170,14cyclictest24233-21timerandwakeup07:30:280
124799163,8cyclictest0-21swapper/307:25:123
124799162,10cyclictest35966irq/58-eth007:45:133
124799161,7cyclictest0-21swapper/307:30:113
122699168,6cyclictest3671-21ntpq08:50:251
122699168,6cyclictest3671-21ntpq08:50:251
122699168,6cyclictest11954-21awk08:25:261
122699167,7cyclictest9963-21irqrtprio11:10:191
122699162,11cyclictest23029-21apt-get08:40:131
122699161,13cyclictest29209-21cut08:45:221
122699161,13cyclictest11084-21/usr/sbin/munin07:20:161
122699161,12cyclictest9764-21kthreadcore07:50:221
122699161,12cyclictest5691-21ps08:55:181
122699161,12cyclictest17122-21kthreadcore07:25:201
121499166,8cyclictest28462-21tr07:35:210
121499163,11cyclictest1151-21kthreadcore08:50:190
121499163,11cyclictest1151-21kthreadcore08:50:190
121499163,11cyclictest10180-21kthreadcore07:50:220
124799157,6cyclictest0-21swapper/307:18:223
124799156,6cyclictest0-21swapper/308:50:283
124799156,6cyclictest0-21swapper/308:50:283
124799152,9cyclictest0-21swapper/307:50:133
124799151,11cyclictest0-21swapper/307:45:023
122699151,4cyclictest0-21swapper/108:15:091
122699151,12cyclictest4235-21kthreadcore07:10:221
121499159,4cyclictest0-21swapper/008:05:300
124799149,3cyclictest0-21swapper/308:55:263
124799141,6cyclictest0-21swapper/308:15:213
124799138,3cyclictest0-21swapper/308:30:213
124799122,7cyclictest0-21swapper/308:25:013
124799122,7cyclictest0-21swapper/307:20:173
124799122,6cyclictest0-21swapper/308:00:143
124799111,7cyclictest0-21swapper/307:10:023
121499111,8cyclictest1251-21sed07:10:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional