You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-15 - 16:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Tue Sep 15, 2026 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8568998520,23cyclictest131rcu_preempt10:15:112
856899606,44cyclictest27621-21ssh12:10:182
8568995615,26cyclictest27-21ksoftirqd/207:35:132
8576995412,26cyclictest35966irq/58-eth010:00:143
854999544,37cyclictest379-21ssh11:55:110
855799511,45cyclictest23851-21apt-get11:50:151
855799511,38cyclictest27300-21ssh10:40:121
855799501,45cyclictest5864-21apt-get11:40:131
855799500,45cyclictest0-21swapper/109:35:151
856899491,26cyclictest25421-21sh09:45:112
855799470,31cyclictest8090-21ssh11:21:451
855799461,39cyclictest0-21swapper/110:10:111
855799451,38cyclictest14578-21apt-get10:15:131
857699443,26cyclictest35966irq/58-eth008:35:153
8568994426,10cyclictest0-21swapper/210:45:112
855799442,36cyclictest25759-21apt-get08:35:151
855799441,37cyclictest18270-21apt-get10:35:151
855799432,36cyclictest32725-21apt-get10:25:141
855799431,37cyclictest0-21swapper/112:00:141
857699425,21cyclictest35966irq/58-eth011:05:133
857699423,23cyclictest35966irq/58-eth009:40:133
855799421,2cyclictest0-21swapper/109:45:111
854999421,3cyclictest131rcu_preempt09:35:130
8576994133,5cyclictest35966irq/58-eth010:20:143
857699413,25cyclictest35966irq/58-eth010:55:143
856899414,28cyclictest27-21ksoftirqd/209:20:122
855799411,36cyclictest0-21swapper/107:10:131
855799411,34cyclictest3839-21apt-get10:45:131
855799411,22cyclictest0-21swapper/109:12:281
855799410,36cyclictest12891-21apt-get10:50:131
856899402,31cyclictest25172-21cron11:15:012
855799401,22cyclictest0-21swapper/111:05:401
855799400,35cyclictest21-21ksoftirqd/108:55:151
855799400,27cyclictest21148-21ssh12:24:161
856899397,16cyclictest5126-21sh11:02:372
856899392,19cyclictest2435-21ssh10:06:392
856899392,19cyclictest2435-21ssh10:06:382
856899391,21cyclictest2184-21ssh11:20:142
855799390,3cyclictest6485-21apt-get08:15:131
854999391,36cyclictest7718-21ssh12:35:130
856899383,27cyclictest21954-21ssh10:55:132
856899382,19cyclictest24872-21ssh10:37:372
855799381,33cyclictest7689-21apt-get12:35:121
855799381,32cyclictest0-21swapper/108:30:131
855799381,32cyclictest0-21swapper/107:50:121
855799381,31cyclictest0-21swapper/108:40:121
854999387,9cyclictest131rcu_preempt08:20:150
854999382,32cyclictest23863-21ssh11:50:170
854999381,33cyclictest32751-21ssh10:25:130
857699376,18cyclictest35966irq/58-eth011:00:143
857699375,20cyclictest35966irq/58-eth012:15:133
857699371,19cyclictest35966irq/58-eth008:05:133
856899372,29cyclictest12367-21ssh12:19:382
855799371,32cyclictest16433-21apt-get09:40:131
855799370,2cyclictest0-21swapper/109:00:131
8576993622,8cyclictest33-21ksoftirqd/311:17:383
856899363,19cyclictest27346irq/40-dwc2_hso10:20:122
856899362,28cyclictest25970-21ssh11:31:362
855799362,29cyclictest18384-21apt-get07:20:131
855799361,31cyclictest0-21swapper/109:25:131
855799360,22cyclictest11417-21ssh09:18:551
854999361,31cyclictest3850-21ssh10:45:120
857699351,21cyclictest35966irq/58-eth010:45:133
855799352,30cyclictest0-21swapper/111:10:151
855799351,31cyclictest11254-21fschecks_time09:55:191
855799351,29cyclictest0-21swapper/111:25:121
855799351,29cyclictest0-21swapper/111:25:111
855799351,22cyclictest25924-21sh10:01:431
855799350,30cyclictest1949-21/usr/sbin/munin09:50:161
857699348,11cyclictest35966irq/58-eth012:10:163
855799341,30cyclictest9310-21apt-get10:30:131
855799341,30cyclictest29457-21fschecks_time11:35:171
855799341,2cyclictest0-21swapper/107:55:151
855799341,26cyclictest22203-21ntp_192.168.11412:05:261
855799341,24cyclictest3683-21ssh12:15:001
855799340,4cyclictest11299-21apt-get08:20:141
854999342,3cyclictest131rcu_preempt08:35:130
854999342,28cyclictest16436-21sh09:40:120
854999341,26cyclictest0-21swapper/009:25:130
856899339,5cyclictest0-21swapper/211:40:132
855799337,19cyclictest21-21ksoftirqd/107:30:111
855799331,30cyclictest23624-21/usr/sbin/munin10:20:171
855799331,2cyclictest0-21swapper/109:20:111
855799331,2cyclictest0-21swapper/108:00:121
855799331,29cyclictest2247-21sed09:30:281
855799331,29cyclictest2149-21ntp_192.168.11411:00:251
855799330,30cyclictest1918-21sed11:20:011
855799322,23cyclictest16110-21apt-get08:25:131
855799321,28cyclictest23974-21kthreadcore12:25:201
855799321,27cyclictest26873-21latency_hist09:10:021
855799321,27cyclictest14843-21cut07:15:201
855799321,26cyclictest0-21swapper/110:05:121
855799321,26cyclictest0-21swapper/110:05:111
855799320,3cyclictest20393-21ssh11:30:181
855799320,28cyclictest28039-21cut07:30:021
857699313,16cyclictest35966irq/58-eth008:10:123
8576993115,10cyclictest35966irq/58-eth010:50:133
856899312,20cyclictest16450-21ssh09:40:122
856899312,14cyclictest0-21swapper/209:59:382
855799314,24cyclictest5424-21cut08:45:201
8557993116,11cyclictest21-21ksoftirqd/110:55:141
855799311,27cyclictest7957-21mailstats12:15:281
855799311,27cyclictest4157-21/usr/sbin/munin11:55:271
855799311,26cyclictest14814-21/usr/sbin/munin11:45:261
855799310,2cyclictest0-21swapper/112:30:161
855799310,2cyclictest0-21swapper/108:10:121
854999313,21cyclictest628-21tcf-agent07:20:140
8576993018,8cyclictest35966irq/58-eth012:25:113
857699301,18cyclictest35966irq/58-eth007:40:123
856899302,11cyclictest13867-21ssh10:14:372
855799301,26cyclictest1454-21munin-run08:10:011
855799301,26cyclictest13820-21sed07:45:261
857699296,13cyclictest35966irq/58-eth010:30:143
856899294,15cyclictest27346irq/40-dwc2_hso08:30:152
856899292,20cyclictest0-21swapper/207:20:152
856899292,17cyclictest27346irq/40-dwc2_hso08:40:152
855799291,24cyclictest1783-21kthreadcore07:35:211
854999292,20cyclictest3768-21ssh09:15:140
8549992918,8cyclictest12-21ksoftirqd/010:15:110
856899283,21cyclictest0-21swapper/207:40:152
856899283,11cyclictest9321-21ssh10:30:132
856899281,16cyclictest27346irq/40-dwc2_hso07:45:112
855799281,24cyclictest7742-21acpi08:50:121
855799281,24cyclictest7742-21acpi08:50:121
855799280,25cyclictest7975-21cut07:40:211
854999285,17cyclictest1984-21ssh09:50:130
854999283,18cyclictest18290-21ssh10:35:140
854999282,20cyclictest14980-21ssh11:08:370
8549992817,5cyclictest12-21ksoftirqd/008:25:120
854999281,3cyclictest131rcu_preempt11:00:140
857699279,12cyclictest0-21swapper/310:14:373
857699272,15cyclictest0-21swapper/309:45:113
8568992711,12cyclictest0-21swapper/212:35:122
854999272,20cyclictest29082-21ssh11:35:050
854999272,20cyclictest18619-21ssh11:28:370
854999272,20cyclictest18619-21ssh11:28:360
854999272,18cyclictest10506-21ssh11:24:370
857699264,12cyclictest0-21swapper/312:35:143
857699263,17cyclictest0-21swapper/309:55:383
856899266,16cyclictest0-21swapper/211:50:162
856899264,11cyclictest27346irq/40-dwc2_hso08:00:132
856899262,9cyclictest25858-21ssh10:01:392
856899262,9cyclictest0-21swapper/208:15:142
856899261,9cyclictest17681-21kworker/u8:209:25:112
8568992612,11cyclictest28790-21cron11:35:022
856899261,14cyclictest27346irq/40-dwc2_hso08:35:152
856899260,17cyclictest16095-21find08:25:122
854999262,19cyclictest17879-21ssh10:50:360
8549992614,9cyclictest15889-21sshd10:32:370
857699254,13cyclictest0-21swapper/310:15:123
857699252,17cyclictest0-21swapper/311:35:053
856899253,17cyclictest27346irq/40-dwc2_hso11:05:132
856899253,15cyclictest26762-21diskmemload11:45:152
856899252,18cyclictest32174-21ssh11:17:372
856899251,18cyclictest11083-21ssh11:25:042
856899251,18cyclictest11083-21ssh11:25:032
854999256,14cyclictest13139-21ssh12:20:150
854999252,18cyclictest26762-21diskmemload12:03:370
857699243,14cyclictest0-21swapper/311:31:373
857699243,13cyclictest0-21swapper/311:15:013
857699242,13cyclictest0-21swapper/311:50:173
8576992410,10cyclictest35966irq/58-eth009:35:143
856899247,8cyclictest13768-21cat12:00:312
856899245,15cyclictest18660-21kthreadcore09:00:212
856899242,18cyclictest26762-21diskmemload10:41:382
856899242,13cyclictest12432-21latency_hist09:20:012
856899241,8cyclictest5269-21ssh10:25:372
8568992412,9cyclictest4317-21ssh12:31:232
856899241,13cyclictest22182-21ssh12:25:122
856899241,11cyclictest11571-21tail08:50:302
856899241,11cyclictest11571-21tail08:50:292
855799241,21cyclictest8571-21latency_hist07:10:021
854999243,15cyclictest23673-21ssh10:20:140
854999242,8cyclictest17638-21ssh09:57:370
854999242,17cyclictest22214-21ssh12:05:250
854999242,15cyclictest14837-21apt-get11:45:130
857699234,10cyclictest0-21swapper/310:05:133
857699234,10cyclictest0-21swapper/310:05:123
857699232,15cyclictest0-21swapper/311:45:143
8576992313,4cyclictest0-21swapper/310:25:143
8576992310,9cyclictest0-21swapper/310:39:483
8576992310,7cyclictest0-21swapper/312:05:143
856899239,6cyclictest11776-21cat09:35:312
856899234,16cyclictest27164-21munin-plugin-st12:10:012
856899232,9cyclictest5258-21kthreadcore09:50:242
856899232,18cyclictest0-21swapper/208:05:142
8568992313,8cyclictest0-21swapper/208:55:172
856899231,20cyclictest7040-21dpkg09:35:022
8568992311,9cyclictest20180-21ssh12:23:072
856899231,11cyclictest13470-21ps10:50:182
854999232,16cyclictest22171-21ssh12:25:120
8549992312,8cyclictest18131-21cron07:20:010
854999231,17cyclictest12744-21ssh09:20:120
856899228,9cyclictest27-21ksoftirqd/207:50:132
856899225,14cyclictest3070-21fschecks_count08:45:152
856899222,17cyclictest12592-21kthreadcore08:20:232
856899222,15cyclictest0-21swapper/211:55:082
856899222,13cyclictest0-21swapper/208:10:142
856899221,9cyclictest29492-21kthreadcore09:10:232
856899221,11cyclictest22921-21kthreadcore07:55:242
854999227,9cyclictest12-21ksoftirqd/007:10:150
854999224,11cyclictest4086-21ssh12:15:130
854999221,16cyclictest30746-21ssh09:30:150
857699212,13cyclictest0-21swapper/312:03:373
856899216,12cyclictest22172-21sort09:05:112
856899212,8cyclictest0-21swapper/207:30:132
856899211,9cyclictest14513-21kthreadcore07:15:192
856899211,11cyclictest8859-21diskstats07:10:162
856899211,11cyclictest26540-21grep07:25:222
854999211,17cyclictest21940-21apt-get10:55:130
854999211,17cyclictest20092-21ssh10:00:150
854999211,15cyclictest20282-21ssh11:30:160
854999210,17cyclictest27875-21latency_hist07:30:010
857699203,12cyclictest0-21swapper/311:28:373
857699203,12cyclictest0-21swapper/311:28:363
857699202,13cyclictest35966irq/58-eth009:25:133
8576992011,4cyclictest0-21swapper/311:55:393
854999203,12cyclictest0-21swapper/009:05:120
854999202,15cyclictest5755-21ssh11:40:060
854999202,13cyclictest32256-21ssh10:40:380
854999201,14cyclictest7929-21cut08:50:170
854999201,14cyclictest7929-21cut08:50:160
857699199,6cyclictest0-21swapper/309:10:313
857699193,11cyclictest0-21swapper/310:41:373
857699192,10cyclictest0-21swapper/308:15:153
857699192,10cyclictest0-21swapper/307:30:123
857699191,13cyclictest0-21swapper/309:00:173
857699191,13cyclictest0-21swapper/307:40:013
857699191,12cyclictest0-21swapper/309:30:283
857699191,12cyclictest0-21swapper/308:50:293
857699191,12cyclictest0-21swapper/308:50:283
8576991910,6cyclictest0-21swapper/311:22:253
856899192,7cyclictest8656-21latency_hist07:10:022
854999198,7cyclictest1902-21wc08:10:170
854999192,11cyclictest4917-21ssh10:09:370
854999192,11cyclictest4917-21ssh10:09:370
854999191,9cyclictest7176-21kthreadcore07:40:210
854999191,14cyclictest5322-21sshd12:32:380
8549991910,6cyclictest0-21swapper/008:40:230
857699189,6cyclictest35966irq/58-eth012:20:133
857699183,7cyclictest0-21swapper/308:45:143
857699182,11cyclictest0-21swapper/309:53:383
857699182,10cyclictest0-21swapper/308:40:143
857699182,10cyclictest0-21swapper/307:55:163
8576991812,3cyclictest33-21ksoftirqd/309:15:193
8576991810,6cyclictest35966irq/58-eth011:40:143
8576991810,6cyclictest0-21swapper/309:20:523
854999187,8cyclictest16657-21cat11:10:160
854999185,10cyclictest30001-21kthreadcore09:10:240
854999185,10cyclictest10023-21grep08:15:270
854999181,9cyclictest844-21munin-node08:55:150
854999181,7cyclictest9206-21ntp_192.168.11410:10:230
854999181,13cyclictest25857-21/usr/sbin/munin11:15:180
8549991810,6cyclictest14662-21cron07:50:010
857699176,8cyclictest0-21swapper/312:31:523
857699172,10cyclictest0-21swapper/307:20:143
857699171,12cyclictest0-21swapper/308:55:233
854999171,14cyclictest22649-21kthreadcore07:55:240
854999171,12cyclictest25381-21acpi09:45:100
854999171,11cyclictest28834-21kthreadcore12:10:250
857699161,8cyclictest0-21swapper/308:30:023
857699161,10cyclictest0-21swapper/308:30:303
8549991611,3cyclictest27768-21apt-get07:30:040
854999161,12cyclictest492-21diskstats07:35:160
854999161,12cyclictest3023-21diskstats08:45:160
854999161,12cyclictest29293-21apt-get08:05:130
854999161,12cyclictest17463-21diskstats09:00:170
854999161,12cyclictest14897-21apt-get07:50:140
854999161,11cyclictest8596-21cat07:10:010
854999161,11cyclictest22067-21kthreadcore08:30:210
857699151,11cyclictest0-21swapper/307:15:283
854999156,6cyclictest12-21ksoftirqd/008:00:140
857699141,6cyclictest0-21swapper/307:45:163
857699138,3cyclictest0-21swapper/307:54:393
857699138,3cyclictest0-21swapper/307:10:583
857699131,7cyclictest0-21swapper/307:25:173
857699131,6cyclictest0-21swapper/308:20:243
857699122,6cyclictest0-21swapper/308:00:153
857699112,7cyclictest0-21swapper/309:05:153
857699101,7cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional