You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-18 - 18:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Wed Mar 18, 2026 12:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31341996334,21cyclictest28366irq/58-eth012:00:113
31341996334,21cyclictest28366irq/58-eth012:00:113
3133399602,46cyclictest27812-21ssh11:40:122
3133399553,32cyclictest20617-21ssh11:00:112
3131099553,47cyclictest3026-21ssh09:20:130
3133399495,19cyclictest29985-21ssh09:35:142
3133399484,36cyclictest2682-21ssh10:50:152
31333994833,6cyclictest27-21ksoftirqd/212:05:122
3131999471,3cyclictest131rcu_preempt12:25:111
3131099452,33cyclictest4460-21ssh11:45:130
3134199443,21cyclictest28366irq/58-eth008:10:133
31333994430,8cyclictest27-21ksoftirqd/211:25:112
31333994414,21cyclictest27-21ksoftirqd/210:05:122
3134199423,26cyclictest28366irq/58-eth011:20:123
3131999427,6cyclictest131rcu_preempt11:45:131
3131099391,26cyclictest7940-21ssh12:05:110
3134199373,16cyclictest28366irq/58-eth012:30:123
3134199362,4cyclictest0-21swapper/311:40:113
3133399361,29cyclictest924-21snmpd08:30:132
31310993610,6cyclictest131rcu_preempt07:20:110
31341993518,11cyclictest0-21swapper/310:52:273
3134199349,14cyclictest28366irq/58-eth010:55:123
3134199345,18cyclictest0-21swapper/312:20:123
3131999342,6cyclictest131rcu_preempt11:50:121
3131999341,22cyclictest0-21swapper/107:10:131
3134199334,17cyclictest28366irq/58-eth009:30:133
3134199333,20cyclictest28366irq/58-eth009:05:123
3134199333,19cyclictest28366irq/58-eth009:20:143
3134199321,19cyclictest28366irq/58-eth010:05:123
3134199321,18cyclictest28366irq/58-eth009:25:123
3131099322,22cyclictest28653-21cat11:20:280
3134199319,14cyclictest28366irq/58-eth010:20:123
3133399314,13cyclictest22346irq/40-dwc2_hso09:05:122
3133399311,17cyclictest9795-21ssh11:30:122
31319993118,9cyclictest17123-21rm10:40:131
3131999311,20cyclictest3035-21ssh09:20:141
3134199301,18cyclictest28366irq/58-eth008:05:123
3133399302,17cyclictest22346irq/40-dwc2_hso07:55:112
3133399302,17cyclictest22346irq/40-dwc2_hso07:55:102
3133399301,5cyclictest131rcu_preempt12:10:292
3131999308,7cyclictest131rcu_preempt11:05:271
3134199297,6cyclictest0-21swapper/311:10:183
3134199292,17cyclictest28366irq/58-eth007:55:103
3134199292,17cyclictest28366irq/58-eth007:55:093
3134199292,15cyclictest28366irq/58-eth010:10:133
3133399292,12cyclictest990-21ssh10:48:272
31333992914,9cyclictest20959-21ssh09:30:122
3133399291,21cyclictest0-21swapper/210:35:132
3133399291,21cyclictest0-21swapper/210:35:122
3134199283,12cyclictest28366irq/58-eth007:45:133
3134199282,7cyclictest0-21swapper/311:05:243
3133399282,21cyclictest26957-21ssh11:39:282
3133399282,21cyclictest19002-21ssh10:58:322
3133399282,11cyclictest32282-21ssh11:24:272
3133399282,10cyclictest20914-21sshd11:53:412
31333992815,7cyclictest22346irq/40-dwc2_hso09:50:122
3131099282,20cyclictest19765-21ssh10:59:290
3131099282,19cyclictest1462-21cat11:05:280
3134199274,14cyclictest28366irq/58-eth010:00:113
3134199272,20cyclictest28366irq/58-eth011:30:133
31341992716,6cyclictest0-21swapper/311:50:103
3134199271,5cyclictest28366irq/58-eth012:25:113
3133399273,14cyclictest22346irq/40-dwc2_hso12:00:132
3133399273,14cyclictest22346irq/40-dwc2_hso12:00:132
3133399272,21cyclictest0-21swapper/208:00:142
3133399272,19cyclictest0-21swapper/212:33:502
3131999272,21cyclictest15525-21ssh09:45:141
3131999272,19cyclictest13533-21sh11:13:291
3131999272,19cyclictest10039-21ssh10:53:331
31319992713,7cyclictest31299-21cyclictest12:35:111
3131099270,3cyclictest131rcu_preempt11:35:120
3134199266,5cyclictest0-21swapper/312:00:013
3134199263,18cyclictest28366irq/58-eth011:35:133
3134199263,16cyclictest0-21swapper/311:00:283
3134199261,5cyclictest0-21swapper/309:40:213
31341992615,6cyclictest0-21swapper/309:50:133
3134199261,12cyclictest0-21swapper/309:35:153
3134199261,10cyclictest0-21swapper/312:15:223
3133399263,14cyclictest22346irq/40-dwc2_hso08:50:132
3133399263,14cyclictest0-21swapper/208:10:142
3133399262,17cyclictest22346irq/40-dwc2_hso07:15:152
31333992613,7cyclictest27-21ksoftirqd/209:25:132
3133399261,20cyclictest22346irq/40-dwc2_hso09:15:132
3133399261,12cyclictest924-21snmpd07:30:142
3133399261,12cyclictest924-21snmpd07:30:132
3131099262,18cyclictest13641-21ssh11:13:360
3131099261,20cyclictest17098-21ssh10:40:110
3131099261,19cyclictest22412-21ssh11:55:120
3134199257,6cyclictest0-21swapper/311:26:253
3134199251,13cyclictest28366irq/58-eth010:40:113
3133399256,8cyclictest27-21ksoftirqd/211:59:502
3133399254,12cyclictest22346irq/40-dwc2_hso11:10:132
31333992515,8cyclictest0-21swapper/211:15:262
31333992515,8cyclictest0-21swapper/210:00:012
31333992515,8cyclictest0-21swapper/210:00:002
31333992514,9cyclictest0-21swapper/212:15:132
31333992513,7cyclictest261rcuc/209:10:102
3133399251,12cyclictest511-21grep07:45:152
3133399251,10cyclictest261rcuc/210:40:122
3131099252,17cyclictest26054-21/usr/sbin/munin10:45:280
3131099251,7cyclictest21009-21grep11:00:160
3131099251,16cyclictest10068-21sh10:00:120
3134199249,3cyclictest0-21swapper/309:57:543
3134199249,3cyclictest0-21swapper/309:57:533
3134199242,5cyclictest0-21swapper/311:16:283
3134199241,6cyclictest0-21swapper/309:10:183
31341992410,3cyclictest0-21swapper/312:08:163
3133399248,6cyclictest131rcu_preempt12:25:502
3133399247,9cyclictest4663-21grep11:45:162
3133399247,14cyclictest3603-21irqrtprio09:20:182
3133399243,15cyclictest22346irq/40-dwc2_hso10:25:132
3133399242,8cyclictest0-21swapper/208:40:122
3133399242,20cyclictest0-21swapper/207:40:132
3133399242,17cyclictest22346irq/40-dwc2_hso08:25:122
3133399242,16cyclictest22346irq/40-dwc2_hso07:25:142
3133399242,15cyclictest0-21swapper/208:35:042
31333992414,8cyclictest0-21swapper/212:35:232
3133399241,18cyclictest12115-21missed_timers07:20:212
3133399241,12cyclictest6195-21ssh12:20:272
31319992412,6cyclictest0-21swapper/109:25:121
3131099241,17cyclictest0-21swapper/007:10:120
3134199235,6cyclictest0-21swapper/310:39:253
3134199235,6cyclictest0-21swapper/310:39:243
3134199233,9cyclictest0-21swapper/311:48:393
3134199233,7cyclictest0-21swapper/308:35:213
3134199232,14cyclictest28366irq/58-eth008:45:143
3134199231,9cyclictest0-21swapper/309:15:303
3133399239,12cyclictest27-21ksoftirqd/210:13:492
3133399231,13cyclictest32269-21uname10:30:172
31333992311,10cyclictest0-21swapper/211:05:262
31319992319,2cyclictest31441-21ssh12:00:121
31319992319,2cyclictest31441-21ssh12:00:121
31319992312,7cyclictest10085-21ssh10:00:141
3131999231,17cyclictest18832-21ssh11:35:141
3131099231,19cyclictest20998-21ls09:10:220
3131099231,12cyclictest0-21swapper/009:50:310
3131099230,18cyclictest13647-21ssh10:20:140
3134199226,7cyclictest0-21swapper/312:35:123
3134199221,7cyclictest28366irq/58-eth010:45:133
31341992213,5cyclictest0-21swapper/312:10:123
3133399223,13cyclictest22346irq/40-dwc2_hso09:00:132
31333992216,4cyclictest27-21ksoftirqd/210:20:172
31333992214,5cyclictest0-21swapper/210:15:162
31333992212,4cyclictest27-21ksoftirqd/208:15:152
31333992211,8cyclictest0-21swapper/208:49:272
3133399221,12cyclictest17522-21kthreadcore09:45:222
3133399221,11cyclictest22346irq/40-dwc2_hso08:05:112
3131999227,5cyclictest10607-21kthreadcore09:00:201
3131999227,5cyclictest0-21swapper/110:29:521
3131999225,14cyclictest2476-21ssh12:20:121
3131999221,16cyclictest24173-21latency_hist08:10:021
3131999221,16cyclictest15224-21ssh11:15:121
3131099225,7cyclictest131rcu_preempt08:05:010
3131099221,2cyclictest131rcu_preempt08:25:140
31310992212,4cyclictest0-21swapper/010:30:130
3131099221,16cyclictest28073-21ssh10:10:130
3131099221,10cyclictest0-21swapper/007:30:020
3134199210,8cyclictest28366irq/58-eth007:30:133
3134199210,8cyclictest28366irq/58-eth007:30:123
3133399218,4cyclictest27-21ksoftirqd/207:10:232
3133399211,9cyclictest27-21ksoftirqd/207:35:152
3133399211,11cyclictest4300-21kthreadcore08:20:202
3133399211,10cyclictest8596-21tr09:40:212
3133399211,10cyclictest13454-21cut10:00:222
3131999211,9cyclictest1547-21latency_hist08:20:021
31319992112,6cyclictest7403-21cron09:00:011
3131999211,16cyclictest20881-21apt-get08:40:101
3131999211,11cyclictest13214-21kworker/1:209:52:411
3131099212,14cyclictest9146-21ssh10:52:270
3131099211,5cyclictest30561-21kthreadcore12:35:200
3131099211,3cyclictest131rcu_preempt10:15:290
3131099211,18cyclictest19425-21ls09:45:290
3134199209,7cyclictest28366irq/58-eth010:25:123
3134199202,7cyclictest0-21swapper/308:15:133
3134199201,12cyclictest0-21swapper/308:00:123
3133399201,9cyclictest134-21systemd-journal07:55:002
31333992011,7cyclictest0-21swapper/208:55:142
3131999208,9cyclictest24939-1kworker/u9:110:10:141
3131999203,11cyclictest0-21swapper/107:35:121
3131999201,9cyclictest20904-21awk12:10:281
3131999201,4cyclictest21-21ksoftirqd/111:22:201
31319992014,4cyclictest31299-21cyclictest09:30:131
31319992013,4cyclictest22362-21/usr/sbin/munin11:55:321
31319992011,6cyclictest14003-21cat10:20:181
31319992011,5cyclictest31653-21sendmail_mailst11:40:251
3131999201,12cyclictest1469-21ls09:35:281
3131099201,9cyclictest19272-21fschecks_time10:05:160
3131099201,4cyclictest0-21swapper/010:25:170
31310992012,7cyclictest12-21ksoftirqd/012:30:130
31310992011,7cyclictest31429-21apt-get12:00:130
31310992011,7cyclictest31429-21apt-get12:00:120
3131099201,15cyclictest5302-21ls07:50:130
3131099201,15cyclictest13410-21ssh11:50:130
3131099201,14cyclictest1100-21ssh09:55:130
3131099201,14cyclictest1100-21ssh09:55:120
3131099201,12cyclictest0-21swapper/007:55:190
3131099201,12cyclictest0-21swapper/007:55:180
3131099200,8cyclictest24873-21ssh09:30:280
3134199192,10cyclictest0-21swapper/307:20:053
3131999199,8cyclictest21-21ksoftirqd/109:40:121
3131999199,7cyclictest5634-21cat09:55:301
3131999199,7cyclictest5634-21cat09:55:291
3131999199,7cyclictest14691-21cat11:30:311
3131999193,9cyclictest0-21swapper/107:20:121
3131999193,6cyclictest14593-21munin-run08:00:011
3131999193,6cyclictest14593-21munin-run08:00:001
3131999193,13cyclictest8144-21ssh10:35:131
3131999193,13cyclictest8144-21ssh10:35:121
3131999192,15cyclictest17630-21kthreadcore08:00:211
3131999192,13cyclictest25944-21ssh12:15:151
3131999191,7cyclictest20735-21df_inode11:00:141
3131999191,14cyclictest0-21swapper/108:35:121
3131999191,13cyclictest24149-21ntp_states12:30:261
3131999191,10cyclictest747-21apt-get11:25:101
31319991910,6cyclictest0-21swapper/107:20:011
31319991910,5cyclictest26010-21irqcore08:45:191
3131099198,2cyclictest131rcu_preempt07:20:020
3131099195,3cyclictest131rcu_preempt12:15:200
3131099191,4cyclictest12243-21ssh10:35:290
3131099191,4cyclictest12243-21ssh10:35:280
3131099191,4cyclictest0-21swapper/011:30:050
3131099191,4cyclictest0-21swapper/011:25:110
3131099191,13cyclictest26504-21ssh09:15:140
31310991910,7cyclictest1-21systemd12:23:440
3131099190,9cyclictest131rcu_preempt08:35:180
3131099190,9cyclictest131rcu_preempt07:35:010
3131099190,9cyclictest131rcu_preempt07:35:000
3134199182,10cyclictest0-21swapper/307:35:123
3134199182,10cyclictest0-21swapper/307:25:133
3134199182,10cyclictest0-21swapper/307:15:153
3134199181,12cyclictest0-21swapper/307:50:163
3133399181,9cyclictest31336-21latency_hist07:10:022
3131999183,12cyclictest19051-21ssh10:05:111
3131999182,13cyclictest31421-21latency_hist07:10:011
3131999182,12cyclictest2838-21ntp_states10:30:251
3131999181,9cyclictest32634-21cron07:45:011
31319991811,4cyclictest8262-21needreboot10:15:261
31319991810,5cyclictest0-21swapper/108:10:261
3131099187,3cyclictest3509-21uname08:55:190
3131099186,4cyclictest15954-21ssh11:15:190
3131099183,11cyclictest0-21swapper/008:15:130
3131099181,6cyclictest13994-21kthreadcore09:25:220
3131099181,5cyclictest0-21swapper/007:40:210
3131099181,5cyclictest0-21swapper/007:35:190
3131099181,2cyclictest131rcu_preempt11:40:130
3131099181,14cyclictest30550-21ls08:50:130
3131099181,12cyclictest12-21ksoftirqd/009:35:040
3131099181,11cyclictest10779-21cat09:40:310
3134199179,6cyclictest0-21swapper/309:45:103
3134199172,10cyclictest0-21swapper/310:15:163
3134199172,10cyclictest0-21swapper/308:50:133
3134199172,10cyclictest0-21swapper/308:25:143
3134199172,10cyclictest0-21swapper/307:40:123
3134199171,13cyclictest0-21swapper/309:00:213
3134199171,13cyclictest0-21swapper/308:30:263
3131999178,7cyclictest21908-21awk07:30:271
3131999178,7cyclictest21908-21awk07:30:261
3131999177,7cyclictest7929-21apt-get12:05:121
3131999177,7cyclictest27975-21cut10:45:191
3131999174,5cyclictest633-21Async12:11:531
3131999172,12cyclictest21146-21ntp_states09:10:261
3131999172,12cyclictest1010-21kthreadcore08:50:211
3131999172,11cyclictest11641-21apt-get10:55:121
3131999171,14cyclictest15325-21kthreadcore07:25:191
31319991711,4cyclictest0-21swapper/108:29:281
3131099172,12cyclictest27792-21kthreadcore08:45:220
3131099172,12cyclictest11459-21diskmemload12:25:130
3131099171,7cyclictest131rcu_preempt12:10:040
3131099171,4cyclictest0-21swapper/008:30:280
3131099171,3cyclictest0-21swapper/008:40:200
3131099170,9cyclictest26016-21kthreadcore08:10:200
3134199168,6cyclictest0-21swapper/310:30:083
3131999166,7cyclictest21-21ksoftirqd/107:50:121
3131999166,7cyclictest11190-21apt-get08:30:121
3131999161,12cyclictest402-21apt-get07:45:131
3131999161,11cyclictest26632-21/usr/sbin/munin09:15:151
31319991610,4cyclictest12729-1kworker/1:2H08:20:131
3131099169,5cyclictest10693-21cut09:00:200
3131099162,11cyclictest19771-21tune2fs08:05:140
3131099161,6cyclictest3393-21kthreadcore07:45:210
3131099161,6cyclictest13868-21kthreadcore09:05:190
3131099161,13cyclictest5311-21cat08:20:260
3134199149,3cyclictest0-21swapper/307:10:253
3134199142,9cyclictest28366irq/58-eth008:20:113
3134199138,3cyclictest0-21swapper/308:42:043
3131099130,10cyclictest31384-21latency_hist07:10:010
3134199122,6cyclictest0-21swapper/308:55:143
3134199101,6cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional