You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-22 - 21:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Sat Aug 22, 2026 12:45:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1414997522,40cyclictest131rcu_preempt08:50:122
140699667,10cyclictest131rcu_preempt10:30:151
1406996511,6cyclictest131rcu_preempt12:35:141
1406996110,8cyclictest131rcu_preempt11:15:141
141499587,35cyclictest131rcu_preempt07:30:142
140699547,36cyclictest73-21kswapd009:50:111
1414995037,6cyclictest27-21ksoftirqd/212:05:142
139899501,39cyclictest5422-21ssh12:20:120
140699471,36cyclictest21551-21ssh10:05:131
140699471,36cyclictest21551-21ssh10:05:131
1398994741,3cyclictest19286-21diskmemload10:10:120
141499460,39cyclictest19811-21sh10:40:132
141499452,35cyclictest20362-21sed10:20:282
140699445,4cyclictest131rcu_preempt11:45:131
1398994425,15cyclictest8926-21ssh09:40:140
140699424,4cyclictest131rcu_preempt11:20:141
142299413,22cyclictest35966irq/58-eth012:15:143
141499411,25cyclictest27125-21ssh11:20:132
140699413,29cyclictest23378-21sh12:30:141
139899416,10cyclictest131rcu_preempt08:25:140
1422994015,15cyclictest35966irq/58-eth012:35:133
141499402,30cyclictest27346irq/40-dwc2_hso11:55:152
141499402,30cyclictest27346irq/40-dwc2_hso11:55:142
1406994018,11cyclictest131rcu_preempt12:15:131
139899403,33cyclictest12802-21ssh11:30:140
139899401,31cyclictest16212-21ssh10:20:130
141499391,29cyclictest0-21swapper/208:30:122
142299388,18cyclictest35966irq/58-eth010:05:133
142299388,18cyclictest35966irq/58-eth010:05:123
141499382,29cyclictest5999-21sh12:36:562
141499382,29cyclictest13108-21irqrtprio10:00:202
1398993830,5cyclictest25223-21sh11:55:140
1398993830,5cyclictest25223-21sh11:55:130
139899372,23cyclictest844-21munin-node09:25:160
140699366,11cyclictest21-21ksoftirqd/110:50:141
1398993614,15cyclictest3773-21ssh11:25:160
1398993614,15cyclictest3773-21ssh11:25:150
141499355,18cyclictest27346irq/40-dwc2_hso08:55:142
139899358,10cyclictest25499-21cut08:40:210
139899356,14cyclictest6423-21cut10:50:210
1414993410,18cyclictest27-21ksoftirqd/209:10:132
140699342,25cyclictest25249-21ssh10:25:141
140699342,24cyclictest20361-21head10:20:291
139899347,10cyclictest7527-21ssh12:38:460
139899347,10cyclictest29736-21ssh11:02:060
139899346,14cyclictest27692-21kthreadcore10:25:220
139899343,23cyclictest12-21ksoftirqd/010:35:120
142299334,18cyclictest35966irq/58-eth012:05:143
1414993310,7cyclictest930-21snmpd08:00:122
139899338,8cyclictest26445-21awk09:50:010
139899338,8cyclictest26445-21awk09:50:000
139899338,10cyclictest5089-21ssh11:05:420
139899336,11cyclictest5287-21ssh11:43:010
139899334,8cyclictest19286-21diskmemload12:25:270
1398993318,10cyclictest0-21swapper/012:30:140
1398993310,12cyclictest18268-21cut07:25:200
139899327,8cyclictest19837-21df_inode12:10:150
139899327,10cyclictest15365-21irqstats09:05:190
139899325,11cyclictest26104-21kthreadcore09:30:220
1398993224,5cyclictest0-21swapper/008:20:120
139899322,21cyclictest28782-21ssh09:15:120
1398993210,10cyclictest32297-21sed12:15:240
141499312,23cyclictest22251-21usage_Anybus11:15:312
139899318,8cyclictest6463-21entropy07:15:150
139899317,10cyclictest26132-21sh09:12:260
139899316,8cyclictest26640-21ssh11:19:570
139899316,10cyclictest7998-21irqstats11:45:190
139899316,10cyclictest5676-21mailstats12:00:280
139899316,10cyclictest4116-21sed09:55:190
139899316,10cyclictest25582-21/usr/sbin/munin10:05:270
139899316,10cyclictest25582-21/usr/sbin/munin10:05:270
139899315,8cyclictest22093-21diskstats08:05:180
139899315,8cyclictest22093-21diskstats08:05:170
139899314,10cyclictest32302-21ssh11:20:540
139899313,20cyclictest28857-21ssh10:45:150
1398993124,4cyclictest26779-21apt-get08:10:130
1398993114,14cyclictest19800-21apt-get10:40:140
139899311,22cyclictest21751-21ssh11:35:130
1398993110,9cyclictest5757-21sed10:30:290
1398993110,9cyclictest21056-21/usr/sbin/munin07:30:170
142299308,16cyclictest0-21swapper/312:30:133
140699302,7cyclictest21-21ksoftirqd/111:40:121
140699302,7cyclictest21-21ksoftirqd/109:00:121
140699301,20cyclictest19286-21diskmemload12:25:141
139899309,7cyclictest7363-21ssh09:38:400
139899306,8cyclictest31805-21ssh09:50:380
139899306,10cyclictest9132-21cut07:50:190
139899306,10cyclictest11650-21sed07:20:190
139899302,20cyclictest13104-21irqrtprio10:00:200
141499297,16cyclictest0-21swapper/210:30:142
141499294,17cyclictest17875-21sh09:45:142
141499294,17cyclictest17875-21sh09:45:142
141499292,22cyclictest15725-21ssh10:19:572
141499292,12cyclictest14393-21ssh12:25:142
141499291,3cyclictest0-21swapper/208:10:142
141499291,23cyclictest7299-21ssh11:45:132
140699295,15cyclictest3436-21ssh09:55:151
140699292,12cyclictest21-21ksoftirqd/109:05:131
139899298,8cyclictest26207-21kthreadcore07:35:210
139899297,10cyclictest14513-21/usr/sbin/munin10:55:250
139899296,10cyclictest16135-21timerwakeupswit07:55:290
139899294,9cyclictest17743-21usage_EthcTask08:30:300
139899292,21cyclictest14893-21ssh10:18:560
1398992921,4cyclictest10815-21apt-get12:05:140
1398992920,3cyclictest19286-21diskmemload09:20:130
1398992920,3cyclictest19286-21diskmemload09:20:120
1398992919,7cyclictest5121-21apt-get08:55:130
1422992813,10cyclictest35966irq/58-eth011:16:563
142299281,17cyclictest35966irq/58-eth010:25:133
142299281,17cyclictest35966irq/58-eth008:40:153
142299281,11cyclictest0-21swapper/311:25:373
142299281,11cyclictest0-21swapper/311:25:363
141499284,16cyclictest0-21swapper/208:15:132
141499282,21cyclictest26744-21ssh10:05:562
141499282,21cyclictest26744-21ssh10:05:552
141499282,19cyclictest0-21swapper/208:40:142
140699288,13cyclictest1390-21cyclictest07:50:141
140699285,4cyclictest131rcu_preempt09:20:131
140699285,4cyclictest131rcu_preempt09:20:131
139899286,7cyclictest5910-1kworker/u9:111:12:090
139899285,8cyclictest10008-21df_inode09:00:150
139899285,7cyclictest31331-21ntp_192.168.11408:45:220
139899284,15cyclictest0-21swapper/011:50:120
142299273,16cyclictest35966irq/58-eth008:00:143
141499275,15cyclictest0-21swapper/212:30:132
141499273,18cyclictest27-21ksoftirqd/208:25:152
141499271,9cyclictest21725-21/usr/sbin/munin11:35:172
141499271,14cyclictest32563-21ssh11:05:162
1406992714,3cyclictest21-21ksoftirqd/112:20:201
1406992711,3cyclictest131rcu_preempt10:35:141
139899276,7cyclictest4351-21cat08:50:300
139899275,7cyclictest21781-21ntp_states08:35:280
142299263,16cyclictest0-21swapper/310:20:293
141499263,17cyclictest27346irq/40-dwc2_hso12:15:132
141499262,19cyclictest0-21swapper/207:55:042
141499261,9cyclictest19286-21diskmemload09:15:232
1414992616,5cyclictest131rcu_preempt09:25:142
141499261,21cyclictest7212-21latency08:20:212
141499261,21cyclictest17081-21kthreadcore07:25:182
140699269,14cyclictest21-21ksoftirqd/109:45:141
140699269,14cyclictest21-21ksoftirqd/109:45:131
140699267,3cyclictest131rcu_preempt11:05:131
140699261,4cyclictest131rcu_preempt08:40:141
140699261,17cyclictest201rcuc/110:40:141
1398992620,3cyclictest17191-21apt-get08:00:130
139899261,12cyclictest32229-21irqstats08:15:190
139899260,15cyclictest31154-21kthreadcore07:40:200
141499259,7cyclictest5670-21fschecks_time10:50:172
141499252,8cyclictest6827-21ssh11:44:562
141499252,17cyclictest0-21swapper/210:55:052
141499252,16cyclictest0-21swapper/212:20:042
141499252,16cyclictest0-21swapper/211:50:142
1414992514,5cyclictest27-21ksoftirqd/209:40:212
141499251,17cyclictest27346irq/40-dwc2_hso07:45:122
141499250,12cyclictest5365-21ssh09:20:142
141499250,12cyclictest5365-21ssh09:20:132
140699254,5cyclictest131rcu_preempt09:10:121
140699252,16cyclictest21-21ksoftirqd/107:45:121
1406992512,3cyclictest21-21ksoftirqd/111:25:211
1406992512,3cyclictest21-21ksoftirqd/111:25:211
1422992414,4cyclictest0-21swapper/311:40:113
141499249,12cyclictest18330-21apt-get08:35:152
141499244,17cyclictest27-21ksoftirqd/207:50:132
141499243,9cyclictest131rcu_preempt11:30:012
141499243,9cyclictest131rcu_preempt11:30:002
141499242,18cyclictest1813-21apt-get12:00:142
141499242,15cyclictest27346irq/40-dwc2_hso10:25:142
141499241,15cyclictest131rcu_preempt09:30:192
140699249,4cyclictest21-21ksoftirqd/111:10:281
1406992413,3cyclictest21-21ksoftirqd/109:30:261
1406992412,10cyclictest21-21ksoftirqd/112:10:231
140699241,17cyclictest32729-21sshd08:50:131
1406992411,3cyclictest21-21ksoftirqd/111:03:111
1406992411,10cyclictest21-21ksoftirqd/110:10:201
1406992410,4cyclictest131rcu_preempt07:15:211
1398992418,3cyclictest2743-21apt-get07:45:130
142299239,9cyclictest35966irq/58-eth010:30:153
142299233,12cyclictest0-21swapper/310:55:043
141499232,6cyclictest0-21swapper/211:33:572
141499232,6cyclictest0-21swapper/211:12:562
141499232,16cyclictest0-21swapper/210:10:562
141499231,9cyclictest32267-21ssh09:35:122
141499231,3cyclictest131rcu_preempt07:15:132
1414992311,9cyclictest21734-21kthreadcore12:10:222
140699237,3cyclictest131rcu_preempt11:35:261
1406992315,5cyclictest21-21ksoftirqd/111:30:281
1406992312,3cyclictest131rcu_preempt09:25:151
140699231,10cyclictest14947-21irqcore10:55:181
1422992214,4cyclictest0-21swapper/311:45:143
141499221,16cyclictest0-21swapper/207:10:112
1414992210,10cyclictest0-21swapper/210:38:462
140699222,3cyclictest19286-21diskmemload10:00:201
140699222,16cyclictest32276-21ssh09:35:121
140699221,4cyclictest131rcu_preempt09:15:121
1406992214,4cyclictest21-21ksoftirqd/108:30:151
1406992213,6cyclictest21-21ksoftirqd/110:45:141
1406992210,3cyclictest21-21ksoftirqd/112:05:131
1406992210,3cyclictest21-21ksoftirqd/110:18:501
1406992210,3cyclictest21-21ksoftirqd/107:10:241
139899227,12cyclictest1685-21fschecks_count07:10:140
142299217,10cyclictest0-21swapper/312:00:123
1422992114,5cyclictest35966irq/58-eth010:00:143
1422992112,7cyclictest35966irq/58-eth009:55:133
1422992110,8cyclictest36066irq/59-eth011:10:183
142299210,9cyclictest36066irq/59-eth010:42:573
141499217,11cyclictest22128-21fschecks_count08:05:182
141499217,11cyclictest22128-21fschecks_count08:05:172
141499213,10cyclictest29442-21irqrtprio10:45:202
141499211,3cyclictest0-21swapper/208:45:132
1414992111,4cyclictest27-21ksoftirqd/209:55:202
141499211,11cyclictest23734-21fschecks_count11:00:162
140699219,10cyclictest0-21swapper/111:58:361
140699219,10cyclictest0-21swapper/111:58:351
140699215,2cyclictest131rcu_preempt08:00:141
140699212,17cyclictest1830-21ssh12:00:141
1406992115,3cyclictest21-21ksoftirqd/108:25:201
140699211,4cyclictest131rcu_preempt09:40:201
1406992114,4cyclictest21-21ksoftirqd/108:15:131
1406992112,5cyclictest21-21ksoftirqd/108:35:221
142299209,6cyclictest0-21swapper/310:13:083
142299202,11cyclictest0-21swapper/308:55:143
1422992011,7cyclictest0-21swapper/312:25:283
142299201,11cyclictest35966irq/58-eth010:15:143
141499201,9cyclictest131rcu_preempt09:05:262
141499201,8cyclictest32530-21kthreadcore07:40:232
141499201,8cyclictest30315-21latency_hist07:40:022
141499201,16cyclictest26973-21fschecks_count09:50:152
141499201,10cyclictest14676-21wc07:20:282
141499201,10cyclictest10400-21irqcore09:00:172
140699209,3cyclictest21-21ksoftirqd/108:20:261
140699202,15cyclictest14985-21tail07:20:291
1406992014,4cyclictest0-21swapper/108:45:171
1406992012,4cyclictest21-21ksoftirqd/107:25:181
140699201,15cyclictest19869-21ssh11:50:251
142299198,9cyclictest0-21swapper/311:39:423
142299197,9cyclictest0-21swapper/311:57:223
142299197,9cyclictest0-21swapper/311:57:213
142299197,9cyclictest0-21swapper/309:42:413
1422991913,4cyclictest0-21swapper/311:25:023
1422991912,4cyclictest0-21swapper/307:45:133
1422991910,6cyclictest0-21swapper/312:20:173
1422991910,6cyclictest0-21swapper/311:03:433
1422991910,6cyclictest0-21swapper/309:54:433
1422991910,6cyclictest0-21swapper/309:25:383
142299190,11cyclictest35966irq/58-eth009:15:123
141499191,8cyclictest1474-21latency_hist07:10:022
140699198,3cyclictest21-21ksoftirqd/107:30:181
1406991912,4cyclictest21-21ksoftirqd/108:10:201
1406991911,5cyclictest0-21swapper/107:55:281
142299189,6cyclictest0-21swapper/311:50:123
1422991811,2cyclictest0-21swapper/308:05:153
1422991811,2cyclictest0-21swapper/308:05:143
1422991810,6cyclictest0-21swapper/309:20:043
1422991810,6cyclictest0-21swapper/309:20:033
1422991810,5cyclictest0-21swapper/309:35:223
140699181,9cyclictest6566-21cut08:55:211
1406991812,3cyclictest21-21ksoftirqd/108:05:231
1406991812,3cyclictest21-21ksoftirqd/108:05:231
1406991810,4cyclictest21-21ksoftirqd/107:35:161
142299179,6cyclictest0-21swapper/310:54:263
142299179,6cyclictest0-21swapper/309:45:473
142299179,6cyclictest0-21swapper/309:45:463
142299179,6cyclictest0-21swapper/309:12:533
142299178,6cyclictest35966irq/58-eth011:30:143
142299178,6cyclictest0-21swapper/310:37:273
142299172,10cyclictest0-21swapper/307:30:143
1422991712,3cyclictest35966irq/58-eth011:10:003
1422991711,4cyclictest0-21swapper/309:30:293
1406991712,3cyclictest21-21ksoftirqd/107:40:191
142299167,6cyclictest35966irq/58-eth012:10:133
142299166,8cyclictest33-21ksoftirqd/308:35:003
142299162,10cyclictest0-21swapper/307:15:143
142299161,8cyclictest0-21swapper/307:55:053
1422991611,4cyclictest35966irq/58-eth010:45:423
142299151,9cyclictest35966irq/58-eth008:25:143
1422991510,3cyclictest0-21swapper/308:22:563
1422991510,3cyclictest0-21swapper/307:52:553
140699152,10cyclictest1463-21latency_hist07:10:031
139899157,6cyclictest0-21swapper/007:10:010
142299141,8cyclictest0-21swapper/307:35:163
142299141,7cyclictest0-21swapper/307:45:013
142299138,3cyclictest0-21swapper/309:07:393
142299136,5cyclictest33-21ksoftirqd/307:21:013
142299132,6cyclictest0-21swapper/308:45:123
142299131,5cyclictest0-21swapper/307:10:283
142299127,3cyclictest36066irq/59-eth008:10:233
142299127,3cyclictest35966irq/58-eth008:15:233
142299123,7cyclictest33-21ksoftirqd/308:35:283
142299123,7cyclictest33-21ksoftirqd/307:25:163
142299123,4cyclictest0-21swapper/308:50:133
142299121,8cyclictest0-21swapper/309:05:023
142299101,6cyclictest0-21swapper/307:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional