You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-05 - 23:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sun Apr 05, 2026 12:45:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
550995627,22cyclictest28366irq/58-eth009:15:103
52299513,39cyclictest31334-21ssh09:35:130
522995110,7cyclictest131rcu_preempt10:00:130
541994838,5cyclictest0-21swapper/209:15:122
54199474,33cyclictest10407-21ssh11:30:112
52299465,3cyclictest131rcu_preempt09:05:130
52299463,33cyclictest26870-21ssh10:45:110
52299452,4cyclictest131rcu_preempt09:45:110
52399444,31cyclictest20041-21ssh10:05:131
541994316,18cyclictest27-21ksoftirqd/211:20:122
54199431,31cyclictest28251-21ssh11:40:122
52299412,31cyclictest4782-21ssh11:45:120
550994019,15cyclictest28366irq/58-eth010:00:113
550993917,15cyclictest28366irq/58-eth011:50:143
541993912,13cyclictest27-21ksoftirqd/210:10:132
52299393,20cyclictest26114-21ssh12:15:100
52299382,24cyclictest21418-21ssh11:00:120
55099373,18cyclictest28366irq/58-eth012:25:143
55099354,18cyclictest28366irq/58-eth009:10:143
54199353,9cyclictest0-21swapper/209:31:202
54199352,26cyclictest26350-21rm10:07:202
54199352,17cyclictest6308-21ssh10:32:212
54199352,17cyclictest3415-21grep07:10:212
54199352,17cyclictest3415-21grep07:10:202
54199352,17cyclictest12107-21ssh10:35:202
541993512,13cyclictest0-21swapper/209:20:122
54199344,24cyclictest633-21Async12:11:532
52299342,27cyclictest2293-21ssh09:55:130
52299341,26cyclictest31768-21ssh12:00:140
52299341,26cyclictest31768-21ssh12:00:130
52299341,23cyclictest22408-21ssh09:30:140
54199331,23cyclictest19387-21sh11:35:152
52399331,24cyclictest0-21swapper/108:30:131
52299332,23cyclictest9700-21kthreadcore12:05:210
52299328,17cyclictest12-21ksoftirqd/011:30:120
52299327,5cyclictest131rcu_preempt07:15:130
52299321,21cyclictest8895-21ssh10:35:100
54199318,15cyclictest27-21ksoftirqd/211:03:182
54199318,12cyclictest27-21ksoftirqd/211:10:202
54199315,14cyclictest22346irq/40-dwc2_hso08:05:142
54199313,17cyclictest22346irq/40-dwc2_hso07:15:122
541993119,7cyclictest261rcuc/209:50:132
52299312,23cyclictest20456-21ssh10:59:200
52299312,22cyclictest8647-21cut10:15:210
52299312,22cyclictest8647-21cut10:15:200
55099305,15cyclictest28366irq/58-eth012:30:123
55099301,18cyclictest28366irq/58-eth009:45:133
541993013,11cyclictest27-21ksoftirqd/211:00:012
52399303,18cyclictest6933-21ssh11:10:121
52299305,19cyclictest21404-21ssh09:29:170
52299302,25cyclictest1462-21ssh11:25:110
52299302,25cyclictest1462-21ssh11:25:100
55099292,14cyclictest28366irq/58-eth009:20:123
54199297,12cyclictest0-21swapper/208:10:142
54199293,20cyclictest0-21swapper/209:26:202
54199291,23cyclictest15890-21ssh11:15:132
54199291,17cyclictest22346irq/40-dwc2_hso10:25:142
541992910,11cyclictest27-21ksoftirqd/212:15:102
52399299,14cyclictest131rcu_preempt11:40:111
52399292,19cyclictest8645-21kthreadcore10:15:211
52399292,19cyclictest8645-21kthreadcore10:15:201
52299292,21cyclictest10016-21ssh12:23:210
52299292,20cyclictest25812-21ssh09:50:150
55099283,16cyclictest0-21swapper/309:31:213
55099283,13cyclictest0-21swapper/311:55:143
54199289,5cyclictest27-21ksoftirqd/209:10:202
54199282,17cyclictest11810-21ssh12:25:132
54199281,16cyclictest22808-21ssh11:55:122
52399282,19cyclictest17567-21kthreadcore10:20:201
52399282,18cyclictest2783-21ssh12:20:131
52299282,22cyclictest10645-21sh10:53:210
54199273,18cyclictest22346irq/40-dwc2_hso08:25:112
54199272,6cyclictest131rcu_preempt10:20:272
54199272,19cyclictest0-21swapper/211:50:142
54199272,19cyclictest0-21swapper/208:50:142
54199272,10cyclictest0-21swapper/210:40:152
54199272,10cyclictest0-21swapper/210:40:142
52399272,19cyclictest15054-21kthreadcore09:25:191
55099265,15cyclictest0-21swapper/311:35:153
55099264,14cyclictest0-21swapper/311:40:123
54199269,11cyclictest27-21ksoftirqd/210:52:552
54199268,8cyclictest27-21ksoftirqd/210:46:292
54199265,13cyclictest22346irq/40-dwc2_hso08:45:132
54199263,7cyclictest0-21swapper/212:20:212
54199262,11cyclictest8283-21ssh12:05:162
54199262,11cyclictest0-21swapper/208:40:142
54199261,22cyclictest19725-21ls10:05:022
54199261,18cyclictest16666-21ls08:00:172
52399262,18cyclictest9701-21kthreadcore12:05:221
52299267,4cyclictest131rcu_preempt08:20:170
52299267,3cyclictest131rcu_preempt09:10:180
52299267,3cyclictest12-21ksoftirqd/007:40:130
52299261,15cyclictest0-21swapper/007:55:140
55099254,13cyclictest0-21swapper/310:55:133
55099253,14cyclictest0-21swapper/310:32:203
550992520,3cyclictest321rcuc/312:19:093
550992520,3cyclictest321rcuc/310:40:493
550992520,3cyclictest321rcuc/310:40:483
550992519,4cyclictest321rcuc/311:29:183
550992519,4cyclictest321rcuc/311:29:183
550992519,3cyclictest321rcuc/312:14:513
550992514,7cyclictest28366irq/58-eth011:30:143
54199259,8cyclictest27-21ksoftirqd/212:03:372
54199259,8cyclictest27-21ksoftirqd/212:03:362
54199254,14cyclictest22346irq/40-dwc2_hso07:40:142
54199254,12cyclictest131rcu_preempt09:05:252
54199253,16cyclictest0-21swapper/212:39:222
541992513,4cyclictest27-21ksoftirqd/210:15:272
541992513,4cyclictest27-21ksoftirqd/210:15:272
52399252,19cyclictest18933-21ssh10:40:191
52399252,19cyclictest18933-21ssh10:40:191
52399251,2cyclictest0-21swapper/110:50:121
52299251,2cyclictest131rcu_preempt08:30:100
550992418,3cyclictest321rcuc/310:36:363
54199247,9cyclictest27-21ksoftirqd/207:30:152
54199247,6cyclictest131rcu_preempt11:47:462
54199247,3cyclictest131rcu_preempt09:55:292
54199247,11cyclictest27-21ksoftirqd/209:37:452
54199242,15cyclictest22346irq/40-dwc2_hso09:00:132
54199241,12cyclictest25577-21kthreadcore07:35:182
523992415,6cyclictest201rcuc/111:15:131
52299246,4cyclictest151rcuc/010:10:220
55099235,11cyclictest28466irq/59-eth010:10:123
55099232,3cyclictest0-21swapper/307:10:143
55099232,3cyclictest0-21swapper/307:10:133
550992313,7cyclictest0-21swapper/311:10:323
54199236,7cyclictest131rcu_preempt12:30:212
54199233,11cyclictest30350-21/usr/sbin/munin11:05:172
541992310,6cyclictest20461-21ntp_192.168.11409:45:242
541992310,4cyclictest27-21ksoftirqd/212:13:482
52399238,10cyclictest8903-21diskmemload10:35:121
52399234,13cyclictest131rcu_preempt12:10:121
523992315,4cyclictest211ksoftirqd/108:40:111
52299235,3cyclictest131rcu_preempt10:40:260
52299235,3cyclictest131rcu_preempt10:40:250
52299232,15cyclictest17569-21kthreadcore10:20:210
52299231,5cyclictest15867-21apt-get11:15:120
52299231,17cyclictest29707-21ssh12:35:130
55099224,12cyclictest0-21swapper/311:15:113
55099223,13cyclictest0-21swapper/310:07:213
55099223,11cyclictest28366irq/58-eth009:50:123
550992217,3cyclictest321rcuc/310:28:553
54199223,15cyclictest13936-21kthreadcore08:30:192
54199222,3cyclictest131rcu_preempt09:40:162
54199221,11cyclictest2415-21cut11:25:182
54199221,11cyclictest2415-21cut11:25:172
52399223,9cyclictest0-21swapper/112:00:131
52399223,9cyclictest0-21swapper/112:00:131
52399222,15cyclictest24847-21apt-get11:20:101
52399221,6cyclictest11110-21apt-get10:00:131
52399221,16cyclictest19069-21rm09:10:141
52399221,13cyclictest18653-21diskmemload12:30:171
52299221,18cyclictest10906-21ls11:10:290
522992210,3cyclictest12-21ksoftirqd/007:29:000
55099213,12cyclictest0-21swapper/311:20:123
550992116,3cyclictest321rcuc/311:00:433
550992115,4cyclictest321rcuc/308:55:223
550992115,4cyclictest0-21swapper/309:59:213
550992115,3cyclictest321rcuc/309:05:243
54199217,5cyclictest15962-21kthreadcore07:25:192
54199213,3cyclictest131rcu_preempt08:55:132
54199213,14cyclictest6958-21fschecks_count07:50:162
54199213,11cyclictest19349-21kthreadcore08:35:202
54199212,10cyclictest4698-21kthreadcore07:45:202
54199211,16cyclictest4000-21kthreadcore08:20:192
54199211,16cyclictest14305-21tail07:20:282
52399218,4cyclictest17166systemd-network10:55:281
52399217,8cyclictest21-21ksoftirqd/110:14:171
52399211,9cyclictest131rcu_preempt11:53:531
523992111,8cyclictest201rcuc/109:15:111
52399211,17cyclictest17358-21latency09:05:221
52299214,3cyclictest0-21swapper/009:41:130
52299212,6cyclictest15010-21kthreadcore11:50:210
52299212,2cyclictest131rcu_preempt10:09:190
52299211,6cyclictest24824-21/usr/sbin/munin11:20:230
52299211,6cyclictest11874-21cpuspeed12:25:140
522992113,6cyclictest8425-21ssh10:34:550
52299211,2cyclictest131rcu_preempt11:05:280
52299210,6cyclictest9410-21cut09:00:150
55099202,13cyclictest0-21swapper/312:35:113
550992014,4cyclictest321rcuc/312:24:223
550992014,4cyclictest321rcuc/308:50:183
550992011,6cyclictest0-21swapper/307:45:223
54199201,10cyclictest31140-21sed08:15:162
54199201,10cyclictest31140-21sed08:15:152
52399208,3cyclictest131rcu_preempt09:50:281
52399208,3cyclictest0-21swapper/108:20:241
52399207,4cyclictest0-21swapper/109:42:561
52399205,9cyclictest0-21swapper/109:55:121
52399205,11cyclictest21-21ksoftirqd/107:45:121
52399204,11cyclictest29697-21ssh12:35:111
52399203,4cyclictest131rcu_preempt09:45:131
52399203,14cyclictest29429-21awk07:40:021
52399201,9cyclictest0-21swapper/110:33:261
52299203,4cyclictest131rcu_preempt08:35:220
52299201,4cyclictest21042-21/usr/sbin/munin12:30:160
52299201,4cyclictest18653-21diskmemload11:55:260
52299201,3cyclictest131rcu_preempt08:45:010
52299201,10cyclictest0-21swapper/011:40:120
55099193,7cyclictest0-21swapper/308:20:123
55099193,12cyclictest28366irq/58-eth010:50:133
55099193,11cyclictest0-21swapper/312:05:153
55099193,11cyclictest0-21swapper/310:46:203
55099193,11cyclictest0-21swapper/310:22:203
55099193,11cyclictest0-21swapper/309:27:193
55099192,11cyclictest28366irq/58-eth009:40:153
55099192,11cyclictest0-21swapper/308:35:153
550991913,4cyclictest321rcuc/310:15:523
550991913,4cyclictest321rcuc/310:15:513
55099191,13cyclictest0-21swapper/308:25:283
55099191,12cyclictest0-21swapper/307:15:143
54199198,9cyclictest659-21latency_hist07:10:022
54199198,9cyclictest659-21latency_hist07:10:012
52399194,11cyclictest0-21swapper/110:45:141
52399192,4cyclictest131rcu_preempt08:45:051
52399191,9cyclictest10387-21latency_hist07:20:011
52399191,9cyclictest0-21swapper/109:35:141
52399191,3cyclictest131rcu_preempt11:59:581
52399191,3cyclictest131rcu_preempt07:10:201
52399191,3cyclictest131rcu_preempt07:10:191
52399191,2cyclictest131rcu_preempt12:26:371
52399191,2cyclictest131rcu_preempt09:30:231
52399191,16cyclictest4498-21apt-get08:55:121
523991911,5cyclictest13941-21ssh11:30:221
52399191,13cyclictest1471-21sh11:25:121
52399191,13cyclictest1471-21sh11:25:111
52399191,10cyclictest0-21swapper/111:46:571
52399190,2cyclictest131rcu_preempt07:20:281
52299194,3cyclictest18653-21diskmemload09:21:100
52299192,3cyclictest131rcu_preempt12:10:280
52299191,9cyclictest0-21swapper/011:35:590
52299191,6cyclictest6166-21cut08:55:210
52299191,6cyclictest20215-21fschecks_time07:30:150
522991912,4cyclictest27929-21/usr/sbin/munin09:15:130
55099188,7cyclictest0-21swapper/309:35:073
55099182,9cyclictest28466irq/59-eth011:05:123
55099182,10cyclictest0-21swapper/308:45:143
55099182,10cyclictest0-21swapper/308:10:153
550991812,4cyclictest0-21swapper/312:01:203
550991812,4cyclictest0-21swapper/312:01:193
550991811,4cyclictest0-21swapper/311:45:083
55099181,12cyclictest0-21swapper/307:50:273
52399188,8cyclictest4729-21fw_packets09:20:151
52399185,3cyclictest131rcu_preempt08:50:171
52399185,10cyclictest1-21systemd08:09:051
52399182,11cyclictest18653-21diskmemload11:04:191
52399181,7cyclictest31532-21cut11:05:201
52399181,2cyclictest131rcu_preempt11:35:061
523991811,5cyclictest2356-21cron12:20:021
52399181,13cyclictest8007-21apt-get08:25:111
52299185,3cyclictest131rcu_preempt08:10:130
52299182,5cyclictest131rcu_preempt08:00:170
52299181,6cyclictest5660-21switchtime07:45:260
52299181,2cyclictest0-21swapper/007:35:170
522991812,4cyclictest0-21swapper/008:15:260
522991812,4cyclictest0-21swapper/008:15:250
52299180,5cyclictest18653-21diskmemload10:25:230
55099172,9cyclictest0-21swapper/307:40:143
55099172,9cyclictest0-21swapper/307:30:143
55099172,10cyclictest0-21swapper/308:05:163
52399171,7cyclictest1076-21sed07:40:291
523991712,3cyclictest21-21ksoftirqd/108:15:111
523991712,3cyclictest21-21ksoftirqd/108:15:101
52399171,13cyclictest26873-21latency10:25:251
52399171,12cyclictest22342-21ls08:40:011
52299172,10cyclictest0-21swapper/008:45:050
52299171,7cyclictest21372-21fschecks_time08:05:180
550991611,3cyclictest0-21swapper/307:35:213
550991610,4cyclictest0-21swapper/307:20:263
550991610,3cyclictest0-21swapper/309:00:213
52399169,5cyclictest11372-21cron07:55:001
52399162,11cyclictest13870-21latency_hist09:05:021
52399161,5cyclictest16192-21munin-run08:00:021
52399161,12cyclictest20045-21apt-get07:30:131
52299163,11cyclictest8835-21kthreadcore08:25:190
52299162,4cyclictest53146usb-storage07:20:060
52299162,2cyclictest0-21swapper/008:50:150
52299161,3cyclictest0-21swapper/007:50:140
52299160,2cyclictest131rcu_preempt07:10:250
52299160,2cyclictest131rcu_preempt07:10:250
550991510,3cyclictest0-21swapper/308:30:533
550991510,3cyclictest0-21swapper/307:57:053
550991510,3cyclictest0-21swapper/307:25:283
52399153,10cyclictest17113-21kthreadcore08:00:181
52399152,10cyclictest27368-21kthreadcore08:10:211
52399151,11cyclictest15264-21cpu07:25:131
52399151,10cyclictest626-21ls07:10:021
52399151,10cyclictest626-21ls07:10:011
55099149,3cyclictest0-21swapper/308:16:123
55099149,3cyclictest0-21swapper/308:16:113
55099141,9cyclictest28366irq/58-eth008:00:233
55099141,11cyclictest0-21swapper/307:10:023
55099141,11cyclictest0-21swapper/307:10:013
52299141,9cyclictest623-21grep07:10:010
52299141,9cyclictest623-21grep07:10:010
55099138,3cyclictest0-21swapper/308:40:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional