You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-19 - 18:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Sun Apr 19, 2026 12:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1301599541,6cyclictest0-21swapper/209:00:122
1301599474,4cyclictest0-21swapper/208:00:132
1301599463,7cyclictest13697-21ssh10:30:152
13005994638,5cyclictest18493-21apt-get11:45:131
1300599451,39cyclictest28065-21ssh10:20:141
1301599441,3cyclictest0-21swapper/207:25:112
13005994336,3cyclictest17992-21apt-get07:15:131
13005994331,9cyclictest11006-21apt-get12:35:141
13005994316,10cyclictest12882-21ssh11:22:511
13005994315,8cyclictest26495-21ssh09:41:271
13005994315,11cyclictest5290-21ssh10:41:561
1302199425,21cyclictest35966irq/58-eth011:05:133
1302199421,25cyclictest0-21swapper/310:10:133
13005994237,2cyclictest26211-21apt-get10:55:111
13005994237,2cyclictest26211-21apt-get10:55:111
13005994215,10cyclictest6335-21ssh11:36:371
1302199414,25cyclictest35966irq/58-eth010:40:123
13021994120,10cyclictest35966irq/58-eth012:15:143
1301599414,9cyclictest27346irq/40-dwc2_hso10:00:112
13005994134,3cyclictest29840-21apt-get09:45:141
13005994134,3cyclictest10959-21apt-get08:15:121
13005994132,4cyclictest2621-21apt-get08:40:111
13005994130,7cyclictest4794-21apt-get07:35:121
13005994127,11cyclictest13095-21apt-get12:00:141
13005994115,8cyclictest9173-21ssh12:33:161
13005994114,8cyclictest1831-21rm11:52:501
13005994112,11cyclictest21807-21ssh11:27:431
13005994112,10cyclictest19504-21idleruntime07:50:171
1299899411,3cyclictest131rcu_preempt10:55:110
1299899411,3cyclictest131rcu_preempt10:55:110
1301599401,9cyclictest19073-21grep11:25:292
1301599401,10cyclictest5711-21sed12:30:272
1301599400,5cyclictest29587-21ssh11:15:142
13005994033,2cyclictest29566-21apt-get11:15:141
13005994016,11cyclictest23462-21ssh12:23:071
13005994015,11cyclictest30468-21ssh10:02:011
13005994015,10cyclictest15791-21ssh09:18:571
13005994014,9cyclictest2932-21fschecks_count09:30:161
13005994012,14cyclictest9325-21kthreadcore09:50:211
1301599391,7cyclictest18023-21ps10:50:192
1300599399,20cyclictest21-21ksoftirqd/112:25:131
13005993932,3cyclictest9659-21apt-get07:40:131
13005993916,8cyclictest19032-21rm10:31:091
13005993916,12cyclictest2260-21ssh12:10:271
13005993915,8cyclictest24734-21ssh10:16:351
13005993915,11cyclictest23188-21ssh10:51:491
1301599383,8cyclictest22855-21usage_Anybus11:45:312
1301599381,8cyclictest26694-21cut07:55:212
13015993813,22cyclictest31849-21ssh10:40:152
1301599381,10cyclictest11685-21kthreadcore10:10:202
13005993820,14cyclictest0-21swapper/108:10:111
13005993816,8cyclictest18791-21ssh09:38:031
13005993815,8cyclictest32336-21ssh11:34:431
13005993815,8cyclictest27018-21ssh11:12:231
13005993815,8cyclictest20582-21/usr/sbin/munin09:20:221
13005993815,7cyclictest4580-21ssh10:25:061
13005993815,10cyclictest13169-21ssh10:45:341
13005993812,9cyclictest28428-21ssh10:36:391
13005993811,9cyclictest26403-21ssh09:25:171
12998993820,14cyclictest0-21swapper/011:05:130
1302199374,21cyclictest35966irq/58-eth009:20:123
1301599372,5cyclictest0-21swapper/211:40:122
1301599371,8cyclictest8635-21sed07:35:282
1301599371,8cyclictest6756-21timerandwakeup11:00:272
1301599371,14cyclictest24496-21kthreadcore12:05:212
1301599371,10cyclictest18824-21grep08:55:202
13005993732,2cyclictest26563-21apt-get09:05:121
13005993715,9cyclictest17261-21ssh11:44:001
13005993715,8cyclictest19197-21mailstats09:55:281
13005993715,7cyclictest31144-21diskmemload12:16:191
13005993714,8cyclictest31058-21ntp_states07:25:241
13005993714,7cyclictest0-21swapper/110:13:501
13005993713,7cyclictest0-21swapper/111:00:411
13005993711,10cyclictest9396-21ssh11:56:081
13005993710,10cyclictest13923-21cut07:10:191
1301599367,5cyclictest32063-21kthreadcore09:45:222
1301599361,9cyclictest25464-21kthreadcore10:35:222
1301599361,9cyclictest18396-21kthreadcore08:20:212
1301599361,9cyclictest0-21swapper/210:15:202
1301599361,8cyclictest25705-21cut12:25:172
13005993622,10cyclictest31520-21ssh09:10:111
13005993614,10cyclictest16713-21sshd08:54:501
13005993612,8cyclictest0-21swapper/110:05:461
1299899369,10cyclictest6634-21ssh11:36:590
12998993627,4cyclictest21784-21apt-get09:00:120
12998993622,12cyclictest24334-21apt-get10:00:110
1301599352,12cyclictest26687-21grep07:20:272
1301599352,10cyclictest16757-21latency_hist09:20:012
1301599351,9cyclictest3171-21irqcore09:30:182
1301599351,10cyclictest24342-21grep11:30:182
13005993528,4cyclictest22776-21apt-get07:20:131
13005993522,10cyclictest11698-21apt-get11:05:121
13005993520,12cyclictest0-21swapper/108:45:141
13005993520,12cyclictest0-21swapper/108:45:141
13005993513,8cyclictest30173-21sort08:35:121
13005993513,7cyclictest0-21swapper/112:07:271
1299899359,9cyclictest27529-21ssh12:06:210
1299899359,10cyclictest2422-21ssh11:53:330
1299899359,10cyclictest16265-21ssh10:49:180
1299899358,9cyclictest5934-21ssh09:49:570
1299899357,9cyclictest2017-21missed_timers12:10:230
12998993530,3cyclictest6133-21apt-get08:10:120
12998993529,3cyclictest28892-21apt-get08:00:130
13021993417,13cyclictest35966irq/58-eth011:35:133
13021993416,11cyclictest35966irq/58-eth010:25:153
1301599347,4cyclictest31144-21diskmemload10:00:002
1301599341,9cyclictest6521-21fschecks_count09:50:152
1301599341,9cyclictest4724-21wc10:25:152
1301599341,9cyclictest20826-21usage_Anybus12:20:302
1301599341,16cyclictest6155-21ssh11:20:152
1301599341,12cyclictest7861-21irqcore12:15:202
1301599340,8cyclictest27807-21if_eth011:50:172
1301599340,8cyclictest20664-21ssh11:10:122
13005993428,3cyclictest14476-21apt-get07:45:131
13005993411,7cyclictest0-21swapper/108:05:011
1299899349,8cyclictest31388-21ssh09:25:580
1299899349,8cyclictest31144-21diskmemload12:19:440
1299899349,7cyclictest31144-21diskmemload12:03:160
1299899349,10cyclictest11141-21ssh11:20:430
1299899348,9cyclictest8104-21ssh09:31:060
1299899348,10cyclictest31047-21cut10:20:210
1299899346,14cyclictest0-21swapper/009:22:120
1299899346,10cyclictest21728-21ssh10:34:240
12998993418,13cyclictest24055-21apt-get11:30:140
1301599339,21cyclictest31144-21diskmemload09:35:132
1301599337,23cyclictest4086-21sh10:24:482
1301599335,5cyclictest1544-21if_err_eth108:05:182
1301599335,5cyclictest1544-21if_err_eth108:05:182
1301599335,25cyclictest0-21swapper/209:20:122
1301599332,9cyclictest31762-21fschecks_time09:10:152
1301599331,9cyclictest29849-21ntp_states09:25:282
1301599331,9cyclictest14474-21kthreadcore08:50:202
1301599331,8cyclictest30109-21find09:05:262
1301599331,6cyclictest21626-21awk07:15:272
1301599331,13cyclictest20919-21irqcore08:25:172
1301599331,11cyclictest5827-21cat10:05:322
1301599330,9cyclictest6226-21diskstats08:10:142
1301599330,9cyclictest32541-21fschecks_count07:30:152
1300599338,8cyclictest26339-21munin-run09:05:011
13005993326,4cyclictest20581-21apt-get08:25:131
1299899338,11cyclictest1888966ssh09:55:240
1299899336,10cyclictest24898-21usage_Anybus11:10:300
12998993313,16cyclictest0-21swapper/009:10:110
1299899330,25cyclictest935-21sh10:05:120
1302199325,14cyclictest0-21swapper/309:55:143
1301599327,3cyclictest0-21swapper/212:37:212
1301599321,9cyclictest21693-21kthreadcore09:40:182
1301599321,8cyclictest4671-21ssh11:35:302
1301599321,8cyclictest19799-21irqrtprio07:50:182
1301599321,10cyclictest8339-21ls10:45:142
1300599328,8cyclictest0-21swapper/108:55:301
13005993226,3cyclictest32391-21apt-get07:30:131
1299899329,9cyclictest22804-21ssh10:35:170
1299899329,8cyclictest31144-21diskmemload09:42:080
1299899328,9cyclictest14534-21rm09:54:280
1299899326,14cyclictest0-21swapper/010:18:210
12998993214,14cyclictest0-21swapper/011:15:140
13021993115,9cyclictest35966irq/58-eth008:05:153
13021993115,9cyclictest35966irq/58-eth008:05:153
1301599311,10cyclictest12612-21munin-plugin-st12:00:012
1301599310,9cyclictest26383-21fschecks_count10:55:152
1301599310,9cyclictest26383-21fschecks_count10:55:142
1301599310,9cyclictest1541-21sort08:35:292
1299899318,8cyclictest17318-21ssh10:13:300
1299899318,10cyclictest20286-21ssh11:25:510
1299899317,9cyclictest31144-21diskmemload12:30:210
1299899317,9cyclictest30130-21cat12:25:310
1299899315,9cyclictest16665-21/usr/sbin/munin07:10:260
1299899315,8cyclictest29971-21ntp_192.168.11409:05:230
1299899315,7cyclictest4832-21ssh10:41:230
12998993118,9cyclictest16479-21apt-get12:20:120
12998993112,14cyclictest0-21swapper/011:45:130
1302199308,18cyclictest0-21swapper/310:54:553
1302199302,18cyclictest35966irq/58-eth010:00:113
1301599301,9cyclictest11624-21rm11:05:092
1301599301,8cyclictest31129-21fschecks_time12:10:162
1301599301,7cyclictest16588-21ntp_192.168.11407:10:242
1301599300,11cyclictest9328-21apt-get07:40:042
13005993019,8cyclictest5116-21head08:05:301
13005993019,8cyclictest5116-21head08:05:301
13005993011,7cyclictest24065-21apt-get07:55:121
1299899309,7cyclictest31144-21diskmemload10:54:410
1299899307,7cyclictest7654-21ntpq11:55:260
1299899306,7cyclictest0-21swapper/010:28:300
12998993015,11cyclictest11957-21diskstats09:35:150
12998993012,15cyclictest0-21swapper/007:15:130
1302199295,14cyclictest35966irq/58-eth009:30:133
1302199292,10cyclictest0-21swapper/310:39:553
1301599291,8cyclictest2728-21grep08:40:142
1301599291,6cyclictest21386-21ssh12:04:432
13005992918,8cyclictest27256-21kthreadcore08:30:221
1299899296,8cyclictest0-21swapper/012:36:440
1299899296,16cyclictest14225-21ssh09:17:030
1299899295,10cyclictest14620-21ssh11:40:460
1302199289,12cyclictest35966irq/58-eth011:40:133
1302199282,16cyclictest35966irq/58-eth011:50:133
1301599280,9cyclictest28221-21kthreadcore08:30:232
13005992817,8cyclictest20551-21latency_hist08:25:011
1299899289,7cyclictest5987-21ntp_192.168.11408:40:220
1299899288,8cyclictest5051-21if_err_eth007:35:160
1299899288,7cyclictest0-21swapper/007:40:190
1299899287,8cyclictest7923-21sed08:45:200
1299899287,8cyclictest7923-21sed08:45:200
1299899287,4cyclictest22869-21diskstats07:20:140
1299899286,7cyclictest14355-21ntp_192.168.11408:15:230
12998992821,4cyclictest12164-21apt-get08:50:120
1299899282,10cyclictest18255-21kthreadcore08:20:220
1302199273,13cyclictest35966irq/58-eth009:45:143
1302199272,15cyclictest35966irq/58-eth011:25:143
13021992711,11cyclictest35966irq/58-eth010:15:143
1301599270,9cyclictest11928-21cron08:50:012
1301599270,9cyclictest11928-21cron08:50:012
1299899277,8cyclictest25706-21irqcore08:30:190
1299899276,8cyclictest11234-21sh11:04:560
1299899276,7cyclictest0-21swapper/007:45:260
1302199268,12cyclictest35966irq/58-eth010:55:123
1302199268,12cyclictest35966irq/58-eth010:55:123
1302199266,15cyclictest35966irq/58-eth010:30:153
1302199263,16cyclictest0-21swapper/309:25:143
1302199263,15cyclictest0-21swapper/310:20:033
1302199262,16cyclictest35966irq/58-eth011:00:133
1302199262,11cyclictest35966irq/58-eth008:25:123
13021992511,9cyclictest35966irq/58-eth010:05:143
1301599250,10cyclictest930-21snmpd07:47:282
13005992512,10cyclictest13077-21cat07:10:021
12998992520,3cyclictest1280-21apt-get08:05:160
12998992520,3cyclictest1280-21apt-get08:05:150
12998992512,9cyclictest20897-21irqcore08:25:170
12998992512,10cyclictest30610-21irqcore08:35:180
1302199246,9cyclictest0-21swapper/311:45:163
12998992414,7cyclictest24390-21cut07:55:170
12998992412,9cyclictest568-21irqstats07:30:180
12998992411,9cyclictest19409-21grep07:50:160
1302199235,11cyclictest0-21swapper/308:40:103
1302199233,14cyclictest0-21swapper/312:00:343
1302199233,14cyclictest0-21swapper/311:30:153
1302199233,13cyclictest0-21swapper/312:05:133
13021992311,9cyclictest0-21swapper/311:20:023
1302199231,16cyclictest35966irq/58-eth011:55:133
1302199231,14cyclictest35966irq/58-eth008:45:153
1302199231,14cyclictest35966irq/58-eth008:45:153
12998992311,9cyclictest20811-21grep08:55:280
1302199224,12cyclictest0-21swapper/312:30:153
1302199223,14cyclictest0-21swapper/311:20:153
1302199223,13cyclictest35966irq/58-eth012:25:143
1302199222,14cyclictest0-21swapper/309:10:123
1301599223,11cyclictest12717-21kthreadcore08:15:192
12998992212,7cyclictest0-21swapper/007:25:100
1302199212,13cyclictest0-21swapper/312:10:143
12998992110,8cyclictest13107-21latency_hist07:10:020
1302199204,11cyclictest0-21swapper/309:40:123
1302199202,11cyclictest0-21swapper/308:10:043
1302199202,11cyclictest0-21swapper/307:20:133
1302199199,5cyclictest0-21swapper/312:35:343
1302199194,10cyclictest35966irq/58-eth010:45:143
1302199192,11cyclictest0-21swapper/307:55:133
1302199191,14cyclictest0-21swapper/312:20:223
1302199191,10cyclictest0-21swapper/309:50:273
1302199189,6cyclictest35966irq/58-eth009:15:133
1302199185,8cyclictest0-21swapper/311:10:123
1302199182,10cyclictest0-21swapper/308:50:133
1302199182,10cyclictest0-21swapper/308:15:133
1302199182,10cyclictest0-21swapper/307:45:143
1302199182,10cyclictest0-21swapper/307:40:143
1302199181,10cyclictest0-21swapper/309:05:023
1301599181,9cyclictest13020-21latency_hist07:10:012
1302199172,10cyclictest0-21swapper/308:30:173
1302199171,11cyclictest0-21swapper/309:35:283
1302199162,9cyclictest0-21swapper/309:05:133
1302199151,9cyclictest0-21swapper/307:10:163
1302199151,11cyclictest0-21swapper/308:55:283
1302199142,8cyclictest0-21swapper/307:25:123
1302199142,7cyclictest0-21swapper/307:15:133
1302199142,10cyclictest0-21swapper/307:35:133
1302199141,7cyclictest0-21swapper/307:30:273
1302199132,8cyclictest0-21swapper/308:05:013
1302199131,7cyclictest0-21swapper/308:20:133
1302199121,9cyclictest0-21swapper/307:50:213
1302199121,8cyclictest0-21swapper/307:10:023
1302199111,8cyclictest0-21swapper/308:35:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional