You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-21 - 16:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Tue Jul 21, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26418996620,12cyclictest27-21ksoftirqd/212:10:152
26409996626,7cyclictest21-21ksoftirqd/112:35:131
2640999603,49cyclictest5682-21ssh10:00:131
26409995440,8cyclictest0-21swapper/112:20:121
2641899524,38cyclictest2203-21ssh11:10:132
26426994833,11cyclictest35966irq/58-eth010:00:143
2639599483,36cyclictest442-21ssh11:45:130
26409994729,5cyclictest21-21ksoftirqd/109:25:141
2642699463,26cyclictest35966irq/58-eth011:35:143
2640999462,36cyclictest21623-21ssh09:15:151
2640999441,35cyclictest4080-21ssh10:35:131
2642699422,26cyclictest35966irq/58-eth007:55:143
2640999421,30cyclictest16684-21ssh12:30:141
2639599426,28cyclictest11273-21ssh11:15:150
2639599411,32cyclictest13043-21ssh10:40:130
2639599411,32cyclictest13043-21ssh10:40:120
26418994014,21cyclictest2423-21kworker/2:110:00:142
2642699392,25cyclictest35966irq/58-eth012:05:143
2640999396,21cyclictest21-21ksoftirqd/110:15:131
2640999391,30cyclictest7635-21ssh10:55:111
2642699383,22cyclictest35966irq/58-eth012:25:123
2642699381,18cyclictest35966irq/58-eth010:55:113
26418993723,8cyclictest29334-21sh11:25:162
2641899371,28cyclictest1891-21ssh09:40:142
2641899371,28cyclictest1891-21ssh09:40:132
26395993610,18cyclictest12-21ksoftirqd/011:05:140
2642699351,27cyclictest35966irq/58-eth011:25:153
2642699343,17cyclictest35966irq/58-eth009:10:143
2642699342,16cyclictest35966irq/58-eth009:30:153
26409993416,6cyclictest21-21ksoftirqd/108:15:131
2639599343,23cyclictest12-21ksoftirqd/012:05:120
2642699332,20cyclictest35966irq/58-eth011:50:143
2642699332,20cyclictest35966irq/58-eth011:50:133
2640999332,26cyclictest27485-21ssh10:30:131
2640999322,21cyclictest23771-21ssh10:10:151
2640999322,21cyclictest23771-21ssh10:10:141
2641899313,20cyclictest18521-21sh11:55:142
2641899312,18cyclictest27346irq/40-dwc2_hso09:20:142
2640999311,16cyclictest27469-21sh12:00:121
26395993118,9cyclictest26576-21apt-get07:10:130
2639599310,22cyclictest21994-21sh10:45:110
26426993018,7cyclictest35966irq/58-eth009:25:133
26426993015,9cyclictest35966irq/58-eth010:30:143
2641899302,23cyclictest7185-21ssh09:25:142
2641899302,20cyclictest19853-21sh09:50:042
2641899301,22cyclictest25676-21ssh12:35:132
2640999302,20cyclictest29336-21sh11:25:151
26395993018,8cyclictest30349-21apt-get08:55:130
26426992915,9cyclictest35966irq/58-eth011:45:143
2642699291,15cyclictest35966irq/58-eth008:25:133
2641899292,23cyclictest12230-21diskmemload12:00:132
2641899291,10cyclictest93066snmpd09:37:482
26395992915,9cyclictest27820-21apt-get07:45:140
26395992915,9cyclictest27820-21apt-get07:45:130
26395992914,11cyclictest25260-21ssh09:35:100
2642699285,14cyclictest0-21swapper/311:20:143
2642699284,16cyclictest0-21swapper/309:40:143
2642699284,16cyclictest0-21swapper/309:40:133
2642699281,13cyclictest35966irq/58-eth011:15:143
2641899282,20cyclictest0-21swapper/212:15:142
2641899281,21cyclictest27501-21diskmemload10:30:142
26409992814,11cyclictest21-21ksoftirqd/109:40:141
26409992814,11cyclictest21-21ksoftirqd/109:40:131
26395992814,9cyclictest31051-21ssh10:50:110
2639599281,22cyclictest10931-21ssh09:45:150
2642699272,15cyclictest0-21swapper/311:05:143
26426992717,8cyclictest0-21swapper/310:35:133
26426992713,10cyclictest35966irq/58-eth011:00:143
2641899272,18cyclictest27346irq/40-dwc2_hso11:40:142
2641899271,20cyclictest21643-21ssh09:15:152
26418992710,12cyclictest27-21ksoftirqd/212:25:132
2639599271,6cyclictest0-21swapper/008:40:150
2639599271,19cyclictest16642-21ssh11:00:140
26395992710,4cyclictest2246766ssh10:25:300
2639599270,19cyclictest12-21ksoftirqd/007:25:140
2642699264,12cyclictest35966irq/58-eth011:10:143
2641899262,16cyclictest27346irq/40-dwc2_hso07:50:112
2641899260,18cyclictest25929-21ls10:45:272
2640999261,19cyclictest18539-21ssh11:55:161
2639599263,5cyclictest12-21ksoftirqd/007:15:160
2639599262,16cyclictest14706-21ssh10:05:140
2639599261,19cyclictest29071-21rm09:55:160
2641899253,13cyclictest27346irq/40-dwc2_hso09:05:112
2640999256,8cyclictest630-21Async09:36:311
2640999256,8cyclictest630-21Async09:36:311
2639599254,5cyclictest12-21ksoftirqd/011:29:510
26395992518,3cyclictest23744-21apt-get10:10:140
26395992518,3cyclictest23744-21apt-get10:10:130
2641899243,17cyclictest27346irq/40-dwc2_hso10:10:142
2641899243,17cyclictest27346irq/40-dwc2_hso10:10:132
2641899242,12cyclictest9610-21ssh11:50:162
2641899242,12cyclictest9610-21ssh11:50:152
2641899241,8cyclictest7654-21ssh10:55:122
2641899241,12cyclictest1792366ssh09:11:032
2639599249,9cyclictest1864-21apt-get09:40:140
2639599249,9cyclictest1864-21apt-get09:40:130
2639599246,6cyclictest12-21ksoftirqd/008:30:310
2639599244,6cyclictest12-21ksoftirqd/012:04:270
2639599244,6cyclictest12-21ksoftirqd/011:35:190
26395992419,2cyclictest13112-21apt-get12:10:140
2639599241,8cyclictest12-21ksoftirqd/011:40:140
26395992416,4cyclictest13365-21apt-get07:30:140
2639599241,17cyclictest16228-21ssh09:30:150
2642699239,10cyclictest0-21swapper/311:40:203
2641899238,12cyclictest0-21swapper/208:20:182
2641899233,18cyclictest14394-21latency_hist08:05:012
2641899231,7cyclictest1633866ssh09:30:152
2641899231,7cyclictest108266ssh11:09:102
2641899231,6cyclictest891066ssh10:19:572
2641899231,5cyclictest1837366rm09:48:402
2641899231,19cyclictest29019-21/usr/sbin/munin09:55:182
2641899231,15cyclictest27346irq/40-dwc2_hso07:45:142
2641899231,15cyclictest27346irq/40-dwc2_hso07:45:132
2641899231,14cyclictest2337166ssh10:10:022
2641899231,12cyclictest16700-21diskmemload12:30:162
26418992310,10cyclictest10761-21vmstat11:30:332
2640999233,13cyclictest0-21swapper/111:10:141
2640999233,10cyclictest21-21ksoftirqd/107:30:131
2640999231,19cyclictest31205-21grep10:50:151
2640999231,18cyclictest18173-21apt-get07:35:121
2640999231,16cyclictest23932-21ssh11:40:161
2640999231,15cyclictest19847-21ssh09:50:041
2639599239,9cyclictest7626-21ssh12:25:120
2639599239,8cyclictest12-21ksoftirqd/009:52:050
2639599233,5cyclictest12-21ksoftirqd/011:10:560
2639599233,4cyclictest151rcuc/010:01:040
2639599233,4cyclictest12-21ksoftirqd/009:11:080
26395992318,4cyclictest151rcuc/009:15:540
2639599231,7cyclictest12-21ksoftirqd/010:35:270
26395992317,4cyclictest151rcuc/010:59:500
2639599231,6cyclictest6000-21ssh09:24:060
2642699229,9cyclictest35966irq/58-eth009:15:153
2642699222,12cyclictest35966irq/58-eth012:20:123
26426992210,7cyclictest35966irq/58-eth010:10:153
26426992210,7cyclictest35966irq/58-eth010:10:143
2641899221,8cyclictest0-21swapper/207:40:012
2641899221,7cyclictest2659966ssh11:22:162
2641899221,6cyclictest2407966ssh10:26:252
2641899221,17cyclictest11176-21cut08:35:162
2641899221,17cyclictest0-21swapper/208:30:162
2641899221,13cyclictest2172166ssh11:00:482
2641899221,11cyclictest10206-21kthreadcore10:20:172
2641899220,6cyclictest1865166ssh11:18:342
2640999221,18cyclictest13375-21ls10:20:281
2639599226,8cyclictest12-21ksoftirqd/012:19:300
2639599224,12cyclictest0-21swapper/008:00:130
2639599222,5cyclictest12-21ksoftirqd/010:20:170
2639599222,5cyclictest12-21ksoftirqd/010:17:570
2641899211,9cyclictest3107166ssh10:50:142
2641899211,8cyclictest1672466ssh10:40:272
2641899211,8cyclictest1672466ssh10:40:262
2641899211,6cyclictest909166ssh10:35:412
2641899211,6cyclictest1068766ssh12:07:352
2641899211,12cyclictest18781-21switchtime11:35:312
2641899211,11cyclictest7666-21head07:20:292
2641899211,10cyclictest5166-21fschecks_count07:55:182
26418992110,8cyclictest32400-21sshd12:20:202
2640999218,9cyclictest25703-21ntp_192.168.11412:15:271
2640999216,3cyclictest131rcu_preempt11:45:241
26409992111,7cyclictest21689-21cron12:15:011
2640999211,15cyclictest11021-21apt-get08:35:141
2639599215,10cyclictest0-21swapper/007:20:140
2639599214,4cyclictest12-21ksoftirqd/012:36:460
2639599214,2cyclictest131rcu_preempt11:31:110
2639599213,4cyclictest12-21ksoftirqd/012:31:220
2639599212,5cyclictest12-21ksoftirqd/011:51:030
2639599212,5cyclictest12-21ksoftirqd/011:51:030
2639599212,14cyclictest7160-21apt-get09:25:120
2639599211,7cyclictest12-21ksoftirqd/011:58:490
2639599211,5cyclictest12-21ksoftirqd/010:30:250
26395992115,4cyclictest151rcuc/012:22:270
26395992113,4cyclictest32603-21apt-get07:50:100
2639599210,7cyclictest11548-21uptime09:05:280
2639599210,17cyclictest1413-21find08:25:130
2642699206,10cyclictest0-21swapper/312:15:193
2642699202,10cyclictest35966irq/58-eth007:50:103
26426992011,5cyclictest0-21swapper/310:20:213
2642699201,13cyclictest35966irq/58-eth009:55:153
2642699201,12cyclictest35966irq/58-eth009:50:143
2642699201,11cyclictest35966irq/58-eth007:35:133
2641899205,12cyclictest8567-21sort07:25:152
2641899203,9cyclictest25813-21irqcore08:50:192
2641899203,15cyclictest23894-21tr07:40:202
2641899203,14cyclictest30458-21grep07:10:292
2641899201,9cyclictest19423-21/usr/sbin/munin08:10:162
2641899201,5cyclictest0-21swapper/207:15:142
2641899201,16cyclictest13347-21/usr/sbin/munin07:30:192
2641899201,15cyclictest20533-21latency_hist08:45:022
2641899200,6cyclictest771566ssh11:48:212
2640999207,9cyclictest0-21swapper/107:50:111
2640999206,3cyclictest131rcu_preempt09:35:231
2640999205,3cyclictest131rcu_preempt09:00:021
2640999205,10cyclictest25638-21acpi11:05:121
2640999203,6cyclictest21-21ksoftirqd/110:25:221
2640999203,3cyclictest131rcu_preempt09:45:511
26409992011,6cyclictest23985-21cron08:15:001
2640999201,14cyclictest9601-21ssh11:30:281
2640999201,13cyclictest14891-21ssh10:05:171
2640999201,10cyclictest16549-21kthreadcore08:40:191
2639599207,7cyclictest0-21swapper/008:50:130
2639599206,5cyclictest0-21swapper/008:35:130
2639599205,3cyclictest0-21swapper/008:05:150
2639599204,5cyclictest12-21ksoftirqd/011:20:190
2639599204,2cyclictest131rcu_preempt09:00:190
2639599202,5cyclictest12-21ksoftirqd/008:10:140
2639599201,5cyclictest12-21ksoftirqd/007:40:200
2639599201,4cyclictest12-21ksoftirqd/008:15:160
2642699199,8cyclictest0-21swapper/309:38:163
2642699199,6cyclictest0-21swapper/310:47:403
2642699192,11cyclictest0-21swapper/308:40:143
2642699191,12cyclictest0-21swapper/311:35:023
2642699191,12cyclictest0-21swapper/308:30:293
2641899194,6cyclictest26385-21cyclictest08:55:202
2641899191,9cyclictest3381-21irqrtprio09:00:182
2641899191,9cyclictest20645-21/usr/sbin/munin08:45:152
2641899191,9cyclictest16789-21kthreadcore08:05:242
2641899191,8cyclictest26137-21kthreadcore08:15:212
2641899190,9cyclictest26493-21latency_hist07:10:022
2640999199,8cyclictest16352-21cat09:30:161
2640999195,3cyclictest12230-21diskmemload10:49:481
2640999193,12cyclictest30606-21ssh09:20:131
26409991912,5cyclictest0-21swapper/107:55:191
26409991911,5cyclictest0-21swapper/108:20:151
2640999191,14cyclictest16733-21ssh10:40:271
2640999191,14cyclictest16733-21ssh10:40:261
2640999191,11cyclictest2715-21ls07:15:291
2639599195,3cyclictest0-21swapper/007:35:120
2639599194,4cyclictest0-21swapper/007:59:270
2642699189,6cyclictest35966irq/58-eth009:45:143
2642699188,6cyclictest0-21swapper/312:14:213
2642699188,6cyclictest0-21swapper/310:53:323
2642699183,10cyclictest0-21swapper/307:25:163
2642699182,10cyclictest0-21swapper/312:35:143
2642699182,10cyclictest0-21swapper/309:05:043
2642699182,10cyclictest0-21swapper/308:55:043
2642699182,10cyclictest0-21swapper/308:20:163
2642699182,10cyclictest0-21swapper/307:40:153
2642699181,13cyclictest0-21swapper/308:00:193
26426991810,6cyclictest0-21swapper/310:26:033
2641899181,9cyclictest2076-21kernelversion08:25:202
2640999188,7cyclictest29280-21grep08:50:291
2640999187,3cyclictest22519-21kthreadcore08:45:221
2640999183,4cyclictest21-21ksoftirqd/111:21:251
2640999182,3cyclictest12230-21diskmemload11:01:041
2640999182,2cyclictest12230-21diskmemload12:06:141
2640999181,8cyclictest10785-21cron08:35:001
2640999181,5cyclictest0-21swapper/112:25:491
2640999181,3cyclictest131rcu_preempt11:50:261
2640999181,3cyclictest131rcu_preempt11:50:251
2640999181,3cyclictest0-21swapper/108:05:161
26409991811,4cyclictest15993-21usage_EthcTask11:15:331
2640999181,14cyclictest15021-21ssh11:35:161
2640999181,14cyclictest12744-21df_inode09:10:141
2642699179,6cyclictest0-21swapper/310:05:033
2642699177,7cyclictest35966irq/58-eth012:30:153
2642699175,7cyclictest35966irq/58-eth010:15:143
2642699173,10cyclictest0-21swapper/311:55:153
2642699172,9cyclictest0-21swapper/308:50:143
2642699172,11cyclictest0-21swapper/312:00:143
2642699171,9cyclictest35966irq/58-eth009:20:143
2642699171,13cyclictest0-21swapper/310:40:213
2642699171,13cyclictest0-21swapper/310:40:213
2642699171,10cyclictest0-21swapper/307:15:293
2640999179,6cyclictest7351-21basename09:05:011
2640999175,9cyclictest1757-21grep08:25:201
2640999173,8cyclictest0-21swapper/107:10:121
2640999171,13cyclictest29044-21apt-get09:55:141
2640999171,12cyclictest14476-21wc08:05:021
2639599172,5cyclictest12-21ksoftirqd/008:20:180
2639599172,10cyclictest0-21swapper/008:45:140
2642699162,10cyclictest0-21swapper/308:15:143
2642699162,10cyclictest0-21swapper/307:30:153
2640999168,5cyclictest12275-21cat07:25:301
2640999164,9cyclictest6037-21kthreadcore07:20:211
2640999164,9cyclictest23582-21irqrtprio07:40:191
2640999161,12cyclictest7639-21acpi09:05:111
2642699151,7cyclictest0-21swapper/307:45:123
2642699151,7cyclictest0-21swapper/307:45:123
2642699151,12cyclictest0-21swapper/309:00:233
2639599151,10cyclictest26534-21latency_hist07:10:020
2640999141,9cyclictest26486-21wc07:10:011
2642699139,2cyclictest33-21ksoftirqd/308:36:393
2642699138,3cyclictest0-21swapper/308:10:013
2642699136,5cyclictest35966irq/58-eth007:20:203
2642699131,9cyclictest0-21swapper/308:10:293
2642699121,5cyclictest0-21swapper/308:45:143
2642699113,6cyclictest0-21swapper/307:10:173
2642699101,6cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional