You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-30 - 09:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sun Aug 30, 2026 00:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1364499652,47cyclictest21967-21ssh23:10:121
1364499572,42cyclictest5754-21ssh22:25:131
1364499421,32cyclictest27335-21ssh00:25:121
13663994122,11cyclictest35966irq/58-eth022:50:143
1365499419,9cyclictest131rcu_preempt23:25:142
1364499381,24cyclictest7655-21ssh23:20:131
13663993728,5cyclictest35966irq/58-eth023:20:133
1365499371,2cyclictest27346irq/40-dwc2_hso00:25:132
1364499369,22cyclictest21-21ksoftirqd/120:10:111
1366399353,24cyclictest35966irq/58-eth021:50:133
13663993218,11cyclictest35966irq/58-eth022:15:133
13663993217,10cyclictest35966irq/58-eth022:30:143
13644993214,11cyclictest0-21swapper/122:40:141
1363699324,25cyclictest7292-21ssh21:50:130
13663993115,10cyclictest35966irq/58-eth000:20:133
1366399311,20cyclictest35966irq/58-eth022:40:143
13644993116,8cyclictest21-21ksoftirqd/120:25:141
1364499304,21cyclictest21-21ksoftirqd/100:05:141
13636993021,3cyclictest12-21ksoftirqd/022:25:140
1366399292,19cyclictest35966irq/58-eth023:15:153
1366399292,19cyclictest35966irq/58-eth023:15:153
13654992917,5cyclictest11174-21ssh23:56:062
1363699294,21cyclictest9079-21ssh21:15:130
1363699291,25cyclictest12969-21ssh23:05:130
13663992811,12cyclictest0-21swapper/323:45:143
1364499282,18cyclictest19932-21ssh22:15:131
13636992819,7cyclictest12-21ksoftirqd/020:05:140
1366399273,16cyclictest0-21swapper/323:13:123
1366399272,17cyclictest0-21swapper/321:45:143
1366399272,15cyclictest35966irq/58-eth021:15:123
1366399272,15cyclictest0-21swapper/322:25:143
13663992714,7cyclictest35966irq/58-eth022:10:133
1365499271,18cyclictest3642-21ssh21:30:122
13644992714,10cyclictest0-21swapper/100:35:141
1363699272,23cyclictest21645-21ssh21:40:140
13636992718,6cyclictest30693-21ssh21:45:150
1366399265,11cyclictest0-21swapper/323:05:123
1366399263,14cyclictest0-21swapper/321:30:143
1366399262,15cyclictest35966irq/58-eth019:25:143
1366399261,14cyclictest35966irq/58-eth021:35:153
1365499263,9cyclictest27-21ksoftirqd/222:40:152
1364499262,18cyclictest32042-21diskmemload22:08:111
1363699267,8cyclictest131rcu_preempt23:30:110
1363699266,18cyclictest32042-21diskmemload22:00:120
13636992620,4cyclictest0-21swapper/020:40:110
1363699261,22cyclictest29218-21ssh23:50:110
1363699261,22cyclictest18318-21ssh00:20:120
1366399258,11cyclictest0-21swapper/300:31:133
1366399253,12cyclictest35966irq/58-eth021:40:133
1366399252,14cyclictest0-21swapper/300:10:113
1365499256,8cyclictest28460-21wc19:25:172
1365499251,18cyclictest27982-21sed19:55:222
1365499251,18cyclictest27982-21sed19:55:222
1364499250,19cyclictest17267-21ssh00:19:131
1363699253,19cyclictest1966-21ssh22:05:130
1363699253,18cyclictest12-21ksoftirqd/020:00:120
1363699251,20cyclictest19860-21apt-get19:50:120
1363699251,20cyclictest19860-21apt-get19:50:110
1366399246,12cyclictest33-21ksoftirqd/322:09:113
1366399242,14cyclictest35966irq/58-eth023:30:113
1366399241,14cyclictest35966irq/58-eth022:45:153
13654992412,9cyclictest19688-21ssh22:50:192
1365499241,19cyclictest23979-21ssh00:05:132
1365499241,16cyclictest0-21swapper/220:10:132
13654992411,10cyclictest14887-21ssh22:46:192
13644992416,6cyclictest0-21swapper/122:10:131
1363699243,17cyclictest9417-21ssh22:45:140
1363699242,19cyclictest3699-21ssh21:10:260
13636992411,9cyclictest27106-21diskmemload21:25:130
13636992411,10cyclictest17285-21ssh00:00:220
1366399235,12cyclictest0-21swapper/323:40:133
1366399233,16cyclictest0-21swapper/323:02:123
13663992311,9cyclictest35966irq/58-eth022:35:133
13663992310,9cyclictest35966irq/58-eth000:35:143
1365499237,6cyclictest131rcu_preempt23:15:502
1365499237,6cyclictest131rcu_preempt23:15:492
1365499231,2cyclictest131rcu_preempt00:35:132
1365499231,15cyclictest0-21swapper/221:05:152
1364499236,12cyclictest13626-21cyclictest00:30:121
1364499231,16cyclictest11310-21ssh23:40:141
1363699234,15cyclictest18087-21ssh21:20:130
1363699232,17cyclictest14760-21ssh22:30:130
13636992313,7cyclictest7268-21ssh21:30:270
13636992313,7cyclictest12-21ksoftirqd/020:20:120
1363699231,19cyclictest27350-21ssh00:25:140
1363699231,17cyclictest11002-21ssh22:10:140
13636992310,9cyclictest11110-21cat23:40:020
1366399223,13cyclictest0-21swapper/322:00:133
1366399222,14cyclictest35966irq/58-eth000:05:133
1366399222,10cyclictest35966irq/58-eth021:55:143
1366399221,16cyclictest35966irq/58-eth020:50:133
1366399221,14cyclictest0-21swapper/300:25:133
1365499229,10cyclictest18364-21cron19:15:022
1365499229,10cyclictest0-21swapper/221:45:142
1365499227,5cyclictest131rcu_preempt23:07:082
1365499227,12cyclictest32566-21ssh22:20:242
1365499225,3cyclictest24789-21kthreadcore23:10:212
1365499222,4cyclictest25391-21kthreadcore21:00:212
1365499221,8cyclictest12577-21cut23:40:202
1365499221,7cyclictest0-21swapper/220:00:042
1365499221,6cyclictest32042-21diskmemload00:13:332
1365499221,6cyclictest0-21swapper/222:15:112
13654992212,8cyclictest32042-21diskmemload23:45:262
13654992210,6cyclictest91-21pr/ttyS021:45:012
1364499221,18cyclictest12929-21find23:05:121
1363699225,14cyclictest18373-21ssh22:50:130
1363699222,17cyclictest23767-21apt-get22:35:120
13636992214,6cyclictest20049-21ssh22:15:150
13636992214,5cyclictest8220-21irqrtprio23:20:190
1363699221,2cyclictest131rcu_preempt19:25:160
13636992212,4cyclictest12-21ksoftirqd/000:30:130
13636992210,3cyclictest131rcu_preempt23:30:020
1366399213,10cyclictest0-21swapper/300:15:153
1366399212,13cyclictest0-21swapper/320:25:133
13663992111,4cyclictest0-21swapper/322:55:153
1365499219,9cyclictest13319-21ssh23:21:332
1365499219,5cyclictest10508-21ssh23:02:342
1365499217,11cyclictest30588-21wc21:25:232
1365499214,10cyclictest18328-21ssh00:20:122
1365499211,6cyclictest8445-21kworker/2:100:16:522
13654992116,3cyclictest930-21snmpd21:19:492
1365499211,4cyclictest5960-21kthreadcore00:30:212
1365499211,4cyclictest32042-21diskmemload22:01:052
1365499211,4cyclictest0-21swapper/222:09:292
1365499211,10cyclictest32042-21diskmemload22:58:312
1365499210,7cyclictest808-21kworker/2:121:13:432
1365499210,7cyclictest131rcu_preempt21:51:182
1364499214,11cyclictest0-21swapper/121:05:141
1364499213,12cyclictest23639-21ssh22:35:031
1364499211,17cyclictest16256-21ls20:50:221
1364499211,15cyclictest18338-21ssh00:20:131
1364499211,15cyclictest18116-21ssh21:20:141
1364499211,13cyclictest538-21cut00:10:141
1363699215,12cyclictest0-21swapper/019:55:120
1363699215,12cyclictest0-21swapper/019:55:110
1363699213,4cyclictest131rcu_preempt00:10:090
1363699213,15cyclictest12-21ksoftirqd/019:35:120
1363699212,7cyclictest131rcu_preempt21:57:100
1363699211,4cyclictest0-21swapper/022:42:350
13636992112,6cyclictest14481-21cron00:00:020
13636992112,6cyclictest10655-21sshd23:02:440
1363699211,16cyclictest27435-21ssh22:55:140
1363699211,14cyclictest12898-21missed_timers00:15:250
1366399209,8cyclictest33-21ksoftirqd/323:50:113
1366399203,12cyclictest0-21swapper/321:10:163
1366399203,10cyclictest0-21swapper/323:55:153
1366399201,12cyclictest0-21swapper/319:55:153
1366399201,12cyclictest0-21swapper/319:55:153
1365499205,6cyclictest5543-21grep20:05:272
1365499201,9cyclictest20212-21irqcore19:50:172
1365499201,9cyclictest20212-21irqcore19:50:172
1365499201,9cyclictest12906-21grep20:50:022
1365499201,9cyclictest12724-21diskstats21:35:162
1365499201,8cyclictest5930-21ssh23:35:242
1365499201,4cyclictest714-21grep19:30:172
13654992011,3cyclictest32042-21diskmemload21:55:292
1365499201,11cyclictest930-21snmpd22:30:462
1365499201,11cyclictest930-21snmpd22:10:142
1365499201,11cyclictest24192-21irqcore22:35:172
1365499201,10cyclictest5759-21ssh22:25:132
1365499201,10cyclictest21145-21latency_hist20:25:012
1365499201,10cyclictest18234-21diskstats21:20:152
1365499200,6cyclictest13812-21kworker/2:200:03:522
1364499203,12cyclictest0-21swapper/121:00:141
1364499202,13cyclictest10667-21ssh23:02:451
13644992012,6cyclictest3043-21basename20:40:011
13644992011,7cyclictest21051-21sshd20:23:081
1364499201,15cyclictest30534-21kworker/u8:223:35:141
1364499201,15cyclictest11496-21missed_timers20:45:241
1364499200,17cyclictest27399-21find22:55:121
1363699207,10cyclictest31206-21tail20:35:150
13636992011,6cyclictest27846-21cat00:05:290
13636992011,6cyclictest12592-21ssh21:35:130
1363699201,15cyclictest29183-21ssh22:20:160
1366399194,11cyclictest0-21swapper/321:25:133
1366399192,12cyclictest0-21swapper/323:35:343
1366399191,14cyclictest0-21swapper/300:05:023
1366399191,12cyclictest0-21swapper/320:00:103
1366399191,10cyclictest35966irq/58-eth019:40:143
13663991910,6cyclictest0-21swapper/321:21:403
13663991910,4cyclictest0-21swapper/320:20:223
1365499191,9cyclictest31013-21tune2fs20:35:132
1365499191,9cyclictest15327-21kthreadcore20:50:212
1365499191,9cyclictest15050-21apt-get19:45:132
1365499191,8cyclictest23187-21idleruntime-cro19:20:012
1365499191,14cyclictest29229-21ssh23:50:122
1365499191,10cyclictest7023-21kthreadcore19:35:192
1365499191,10cyclictest29855-21grep20:30:282
1365499191,10cyclictest25534-21acpi23:30:102
1365499191,10cyclictest24667-21tr19:20:192
1365499191,10cyclictest22919-21kthreadcore20:25:192
1365499191,10cyclictest19154-21kthreadcore20:55:182
1365499191,10cyclictest14921-21latency_hist19:45:022
1364499199,7cyclictest499-21sh23:50:251
1364499192,11cyclictest14742-21rm22:30:121
1364499191,13cyclictest32042-21diskmemload21:25:131
1364499191,12cyclictest23616-21ls19:50:281
1364499191,12cyclictest23616-21ls19:50:281
13644991910,6cyclictest20118-21latency23:25:241
1363699199,8cyclictest22315-21cron21:00:010
1363699197,7cyclictest12-21ksoftirqd/019:10:130
1363699192,14cyclictest11837-21timerandwakeup20:45:290
1363699191,9cyclictest0-21swapper/023:10:390
1363699191,14cyclictest753-21sendmail-mta19:30:150
1363699191,14cyclictest32041-21apt-get21:10:020
1363699191,13cyclictest15424-21ssh23:40:300
13636991910,7cyclictest5680-21sshd23:18:120
13636991910,7cyclictest5680-21sshd23:18:120
1366399189,6cyclictest35966irq/58-eth023:25:133
1366399182,10cyclictest0-21swapper/321:00:143
1366399182,10cyclictest0-21swapper/320:05:043
1366399182,10cyclictest0-21swapper/319:10:133
13663991810,5cyclictest35966irq/58-eth022:20:133
1365499184,7cyclictest3432-21df_inode20:40:142
1365499181,9cyclictest13769-21latency_hist19:10:012
1365499181,9cyclictest12597-21kthreadcore20:15:192
1364499189,7cyclictest10692-21cat23:55:341
1364499186,5cyclictest131rcu_preempt21:10:201
1364499183,4cyclictest131rcu_preempt19:25:161
1364499181,9cyclictest17779-21apt-get20:55:101
1364499181,9cyclictest13037-21cut21:35:191
1364499181,2cyclictest131rcu_preempt23:35:011
13644991812,4cyclictest16907-21irqrtprio21:55:191
13644991811,4cyclictest0-21swapper/122:20:041
1364499181,12cyclictest27951-21sed20:30:211
1364499181,12cyclictest19114-21grep22:50:191
1364499181,11cyclictest9480-21ssh22:45:151
13644991810,6cyclictest22717-21cut21:40:201
13644991810,6cyclictest19607-21sshd19:47:081
13644991810,5cyclictest27298-21kthreadcore22:00:201
1363699188,7cyclictest11714-21diskstats20:15:150
1363699187,8cyclictest13048-21df_inode20:50:140
1363699182,13cyclictest12817-21apt-get00:35:130
13636991810,6cyclictest20987-21ssh23:45:200
1366399172,9cyclictest0-21swapper/320:55:123
1366399172,10cyclictest0-21swapper/320:45:143
1366399171,11cyclictest0-21swapper/320:35:023
1366399171,10cyclictest0-21swapper/320:15:283
1364499178,7cyclictest4374-21kernelversion21:30:201
1364499177,8cyclictest5433-21apt-get19:35:121
1364499173,4cyclictest29877-21sed20:00:151
1364499172,11cyclictest7408-21diskstats21:50:151
1364499172,10cyclictest0-21swapper/100:00:051
1364499171,13cyclictest1898-21ntp_192.168.11421:45:261
1364499171,13cyclictest1712-21apt-get20:05:041
1364499171,11cyclictest21-21ksoftirqd/123:45:131
13644991710,4cyclictest25605-21cut19:55:181
13644991710,4cyclictest25605-21cut19:55:181
1364499170,14cyclictest558-21apt-get19:30:131
1363699178,6cyclictest14811-21cron19:45:010
1363699177,7cyclictest18603-21apt-get19:15:140
1363699175,9cyclictest25850-21latency_hist20:30:010
1363699174,10cyclictest23573-21fschecks_count19:20:150
1366399164,10cyclictest35966irq/58-eth019:50:123
1366399164,10cyclictest35966irq/58-eth019:50:123
1366399162,9cyclictest0-21swapper/320:10:133
1366399162,11cyclictest0-21swapper/319:20:133
1366399162,10cyclictest0-21swapper/321:05:143
1364499168,6cyclictest22276-21threads19:15:291
1364499164,4cyclictest131rcu_preempt19:10:131
1364499162,11cyclictest3017-21apt-get20:40:031
1364499162,10cyclictest0-21swapper/119:20:121
1364499161,12cyclictest16231-21idleruntime-cro20:20:011
1364499161,11cyclictest31144-21diskstats23:15:161
1364499161,11cyclictest31144-21diskstats23:15:161
1363699167,7cyclictest6926-21df_inode20:10:140
1363699163,10cyclictest27162-21cron21:05:010
1363699162,11cyclictest26379-21if_eth120:30:170
1363699161,12cyclictest15197-21fschecks_count19:45:150
1366399152,9cyclictest0-21swapper/320:40:043
1366399152,11cyclictest0-21swapper/320:35:123
1366399151,7cyclictest0-21swapper/319:35:013
1364499159,4cyclictest0-21swapper/121:15:131
1364499151,11cyclictest12186-21tr19:40:211
1363699158,5cyclictest0-21swapper/019:10:010
1366399141,7cyclictest0-21swapper/319:45:293
1366399131,4cyclictest0-21swapper/319:15:213
1366399122,7cyclictest0-21swapper/319:10:013
1366399121,8cyclictest0-21swapper/319:35:153
1364499120,9cyclictest13549-21latency_hist19:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional