You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-09 - 17:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Thu Apr 09, 2026 12:45:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2385399726,5cyclictest131rcu_preempt08:55:140
2385399726,5cyclictest131rcu_preempt08:55:140
2385399706,9cyclictest131rcu_preempt09:35:130
2386999693,38cyclictest2954-21ssh12:05:122
2386999693,38cyclictest2954-21ssh12:05:122
2385399582,7cyclictest131rcu_preempt12:25:140
23869995230,9cyclictest27-21ksoftirqd/210:20:112
2386199521,37cyclictest26304-21ssh12:00:111
2385399523,10cyclictest13358-21ssh09:30:110
2386999471,29cyclictest13373-21ssh09:30:122
2385399461,8cyclictest131rcu_preempt08:00:120
2386999453,30cyclictest4278-21ssh09:25:122
23861994319,18cyclictest21-21ksoftirqd/110:45:131
2386199431,30cyclictest25141-21ssh10:30:111
2385399432,31cyclictest31581-21ssh09:40:130
2386999423,32cyclictest0-21swapper/211:28:482
2385399412,32cyclictest11980-21ssh12:10:110
2387799392,20cyclictest28366irq/58-eth009:20:153
2386199395,28cyclictest15670-21ssh12:30:141
2386999382,19cyclictest4239-21ssh09:40:482
2386999381,5cyclictest131rcu_preempt08:30:122
23869993810,12cyclictest27-21ksoftirqd/211:51:482
2385399382,23cyclictest29934-21ssh12:20:120
2386999372,5cyclictest92446snmpd08:15:132
23861993718,8cyclictest21-21ksoftirqd/109:15:121
23877993514,18cyclictest0-21swapper/310:53:483
2386999356,10cyclictest27-21ksoftirqd/211:05:132
2386199351,24cyclictest21294-21ssh10:10:131
2386999344,14cyclictest261rcuc/210:55:112
2386999344,14cyclictest261rcuc/210:55:112
2385399342,27cyclictest13466-21ssh11:35:140
2386999332,15cyclictest9263-21cat07:25:192
2385399332,23cyclictest3556-21ssh10:52:480
2386999322,24cyclictest3071-21ssh10:00:122
23869993214,11cyclictest27-21ksoftirqd/208:05:142
23861993226,4cyclictest0-21swapper/111:10:131
2386199322,24cyclictest9975-21ssh09:10:131
2385399322,22cyclictest14233-21awk07:30:240
2387799312,23cyclictest28366irq/58-eth011:10:133
23869993113,12cyclictest22456-21ssh09:35:132
23877993015,9cyclictest28366irq/58-eth012:00:123
2386999303,22cyclictest0-21swapper/212:20:502
2386999302,21cyclictest0-21swapper/207:50:132
2386999302,15cyclictest6607-21ssh12:25:152
2386999302,14cyclictest24716-21ssh12:35:132
2386199308,13cyclictest29132-21ssh10:50:111
2386199303,8cyclictest0-21swapper/108:15:131
2386199302,21cyclictest3029-21ssh11:28:481
2385399303,20cyclictest4432-21ssh11:30:110
2387799292,17cyclictest28366irq/58-eth011:40:143
2386999294,17cyclictest22346irq/40-dwc2_hso08:00:122
2386999292,21cyclictest24368-21ssh09:16:032
23869992916,10cyclictest0-21swapper/211:40:132
2386199292,21cyclictest9579-21diskmemload12:05:111
2386199292,21cyclictest9579-21diskmemload12:05:111
2386199292,21cyclictest8206-21ssh09:45:131
2386999286,4cyclictest131rcu_preempt07:55:132
23869992813,12cyclictest27-21ksoftirqd/210:40:132
2386999281,18cyclictest9969-21ssh09:10:122
23853992819,5cyclictest24715-21sh12:35:130
2387799272,16cyclictest28366irq/58-eth012:05:113
2387799272,16cyclictest28366irq/58-eth012:05:113
23877992715,6cyclictest0-21swapper/311:50:133
2386999274,15cyclictest22346irq/40-dwc2_hso10:05:132
2386999274,13cyclictest22346irq/40-dwc2_hso08:10:122
2386999273,16cyclictest22346irq/40-dwc2_hso12:00:112
2386999273,10cyclictest17230-21ssh11:55:122
23869992711,6cyclictest27-21ksoftirqd/210:15:132
2386199272,19cyclictest10946-21ssh10:55:481
2386199272,19cyclictest10946-21ssh10:55:481
23861992714,8cyclictest0-21swapper/110:40:141
23861992713,9cyclictest21-21ksoftirqd/110:20:121
2385399272,19cyclictest8641-21ssh10:37:480
2385399272,19cyclictest26000-21ssh10:46:480
2385399272,19cyclictest19402-21ssh10:44:480
2385399272,16cyclictest17246-21ssh11:55:130
2385399271,21cyclictest26818-21kworker/u8:210:00:120
2387799263,13cyclictest28366irq/58-eth009:15:123
2386999263,16cyclictest22346irq/40-dwc2_hso07:40:132
2386999263,16cyclictest0-21swapper/209:20:142
2386999262,9cyclictest0-21swapper/211:03:482
2386999262,9cyclictest0-21swapper/209:50:112
2386199262,19cyclictest26477-21sh09:55:121
2386199261,19cyclictest16029-21ssh10:25:141
2385399263,16cyclictest12-21ksoftirqd/011:20:100
2387799253,13cyclictest28366irq/58-eth010:10:113
2387799251,17cyclictest0-21swapper/309:30:483
2387799251,16cyclictest28366irq/58-eth007:40:133
2386999255,12cyclictest22346irq/40-dwc2_hso08:40:122
2386999255,12cyclictest22346irq/40-dwc2_hso08:40:122
2386999252,14cyclictest22346irq/40-dwc2_hso08:20:132
2386999251,14cyclictest18378-21basename11:20:012
2386199258,11cyclictest0-21swapper/107:30:121
2386199253,16cyclictest21-21ksoftirqd/111:35:131
2385399253,15cyclictest12-21ksoftirqd/012:05:110
2385399253,15cyclictest12-21ksoftirqd/012:05:110
2385399252,18cyclictest8102-21ssh11:50:130
23853992517,3cyclictest531-21usb-storage09:50:100
23853992516,5cyclictest14960-21ssh11:00:140
2385399251,16cyclictest839-21munin-node07:10:140
2386999242,19cyclictest0-21swapper/207:15:132
2386999242,19cyclictest0-21swapper/207:15:132
2386999242,17cyclictest0-21swapper/212:34:502
2386999242,15cyclictest0-21swapper/207:20:112
23869992411,5cyclictest625-21/usr/sbin/munin11:10:152
2386199242,7cyclictest131rcu_preempt11:51:481
2386199241,18cyclictest14970-21ssh11:00:151
2385399249,9cyclictest0-21swapper/011:45:120
2385399241,18cyclictest1204-21apt-get07:20:110
2387799236,11cyclictest0-21swapper/310:37:483
2387799233,13cyclictest0-21swapper/311:28:483
23877992310,8cyclictest28366irq/58-eth009:55:123
2386999232,16cyclictest0-21swapper/211:22:482
2386999232,16cyclictest0-21swapper/209:45:482
2386999232,15cyclictest0-21swapper/207:10:132
23869992312,9cyclictest18512-21ssh11:35:452
23869992312,4cyclictest0-21swapper/209:55:112
2386999231,19cyclictest4998-21apt-get09:05:132
2386999231,13cyclictest73-21kswapd010:45:142
2386999231,12cyclictest30008-21latency_hist10:15:012
23869992310,5cyclictest27-21ksoftirqd/210:35:522
2386199233,13cyclictest1845-21ssh10:35:141
2386199232,17cyclictest0-21swapper/109:36:481
2385399231,5cyclictest7331-21if_eth010:20:160
2387799227,8cyclictest0-21swapper/308:05:133
2386999228,12cyclictest4428-21apt-get11:30:112
2386999222,15cyclictest0-21swapper/210:50:482
2386999222,15cyclictest0-21swapper/207:35:272
2386999222,14cyclictest22346irq/40-dwc2_hso07:30:112
23869992211,9cyclictest28628-21ssh12:18:562
23869992211,8cyclictest0-21swapper/208:45:282
23869992211,8cyclictest0-21swapper/208:45:282
2386999221,16cyclictest18000-21kthreadcore10:25:212
2386199225,10cyclictest0-21swapper/108:00:121
2386199224,12cyclictest201rcuc/111:40:131
2386199222,16cyclictest30378-21sed08:20:281
2386199221,16cyclictest24180-21ssh11:05:161
2385399226,12cyclictest12-21ksoftirqd/011:17:480
2385399220,17cyclictest9463-21ls11:15:020
2387799219,5cyclictest28366irq/58-eth010:00:133
2387799213,12cyclictest0-21swapper/309:35:483
23877992110,8cyclictest0-21swapper/311:46:353
23877992110,7cyclictest28366irq/58-eth011:20:113
2386999216,12cyclictest261rcuc/208:35:122
2386999213,9cyclictest26706-21wc08:50:272
2386999211,9cyclictest1447-21latency_hist10:35:022
2386999211,17cyclictest12131-21cut12:10:152
2386999210,12cyclictest28223-21kthreadcore07:45:212
2386999210,10cyclictest2623-21missed_timers11:45:222
2386199219,9cyclictest17392-21apt-get09:50:101
2386199219,6cyclictest21732-21kthreadcore07:40:211
2386199211,8cyclictest13335-21sed11:15:241
23861992111,7cyclictest4763-21cron09:05:011
2386199211,17cyclictest31979-21kthreadcore08:25:191
2386199211,16cyclictest29940-21ssh12:20:131
2386199211,11cyclictest2266-21cut11:45:211
2386199211,10cyclictest751-21kthreadcore07:50:201
23861992110,8cyclictest10640-21sed12:25:301
2385399218,10cyclictest12-21ksoftirqd/010:25:130
2385399213,15cyclictest27838-21ssh11:25:120
2385399212,14cyclictest10942-21sh10:55:480
2385399212,14cyclictest10942-21sh10:55:480
23853992112,7cyclictest3890-21cron09:25:010
23853992112,6cyclictest15558-21apt-get07:35:120
23853992111,7cyclictest30067-21sendmail_mailqu12:00:260
2385399211,16cyclictest30353-21ssh10:15:140
2387799202,14cyclictest0-21swapper/311:35:133
23877992013,4cyclictest0-21swapper/309:05:173
2386999201,3cyclictest0-21swapper/208:55:132
2386999201,3cyclictest0-21swapper/208:55:132
2386999201,10cyclictest2162-21latency08:25:222
2386199208,3cyclictest131rcu_preempt10:15:201
2386199201,8cyclictest16765-21apt-get11:55:021
23861992013,4cyclictest21-21ksoftirqd/109:40:261
2386199201,17cyclictest28801-21ls07:45:271
2386199201,15cyclictest25035-21sed12:15:291
23861992010,7cyclictest9889-21unixbench_singl07:25:261
2386199200,15cyclictest4818-21tr11:30:171
2386199200,15cyclictest31984-21ls09:20:281
2385399206,3cyclictest131rcu_preempt10:35:020
2385399201,8cyclictest22945-21switchtime09:15:270
2385399200,17cyclictest28384-21sed08:20:200
2385399200,12cyclictest839-21munin-node08:50:140
2387799193,9cyclictest0-21swapper/307:30:243
2387799193,11cyclictest0-21swapper/312:34:503
2387799193,11cyclictest0-21swapper/310:43:483
2387799193,11cyclictest0-21swapper/310:26:483
2387799193,11cyclictest0-21swapper/310:19:483
2387799193,11cyclictest0-21swapper/310:09:483
2387799193,11cyclictest0-21swapper/309:44:483
2387799193,11cyclictest0-21swapper/309:14:483
23877991911,6cyclictest0-21swapper/309:25:123
2387799191,13cyclictest0-21swapper/311:05:173
2386999192,8cyclictest23944-21latency_hist07:10:022
2386999191,8cyclictest761-21irqrtprio09:00:172
2386199193,12cyclictest12320-21grep08:35:281
2386199191,3cyclictest131rcu_preempt12:35:121
2386199191,16cyclictest8257-21ls09:05:231
2386199191,15cyclictest14809-21kthreadcore08:05:211
2386199191,15cyclictest13638-21kthreadcore10:05:211
2386199191,14cyclictest28829-21apt-get07:15:121
2386199191,14cyclictest28829-21apt-get07:15:111
2386199191,13cyclictest6696-21ntp_offset10:00:241
2386199191,10cyclictest19239-21irqcore11:20:161
2385399193,9cyclictest0-21swapper/007:25:190
2385399193,11cyclictest12172-21ssh10:05:140
2385399192,12cyclictest20949-21ssh12:15:130
23853991911,6cyclictest929-21sshd07:16:380
23853991911,6cyclictest929-21sshd07:16:380
2385399191,14cyclictest12238-21ssh09:45:270
2385399190,14cyclictest28454-21latency07:45:210
2387799189,6cyclictest28366irq/58-eth010:20:123
2387799189,5cyclictest28366irq/58-eth012:20:133
2387799188,7cyclictest0-21swapper/312:10:203
2387799186,9cyclictest0-21swapper/311:00:253
2387799183,11cyclictest0-21swapper/310:48:483
2387799183,10cyclictest0-21swapper/307:35:273
2387799182,11cyclictest0-21swapper/311:17:483
2387799182,11cyclictest0-21swapper/309:45:483
2387799182,10cyclictest0-21swapper/308:15:143
2387799182,10cyclictest0-21swapper/308:00:143
2387799182,10cyclictest0-21swapper/307:25:123
23877991812,4cyclictest0-21swapper/312:35:273
23877991812,4cyclictest0-21swapper/312:15:283
23877991811,5cyclictest28366irq/58-eth010:55:103
23877991811,5cyclictest28366irq/58-eth010:55:103
23877991811,4cyclictest0-21swapper/312:25:323
2387799181,13cyclictest0-21swapper/308:20:153
2387799181,10cyclictest28366irq/58-eth011:30:133
2386199189,6cyclictest3118-21kthreadcore07:20:181
2386199188,8cyclictest17504-21grep09:30:271
2386199188,7cyclictest15095-1kworker/1:0H07:35:181
2386199181,6cyclictest8107-21mailstats09:25:261
23861991813,3cyclictest0-21swapper/107:55:291
23861991812,4cyclictest0-21swapper/107:10:021
2386199181,14cyclictest13332-21apt-get08:40:111
2386199181,14cyclictest13332-21apt-get08:40:111
2386199181,14cyclictest11989-21ssh12:10:121
2385399188,7cyclictest13076-21cron08:40:010
2385399186,3cyclictest131rcu_preempt09:55:250
2385399185,3cyclictest131rcu_preempt09:25:200
2385399182,11cyclictest21276-21rm10:10:120
2385399182,11cyclictest0-21swapper/009:05:150
2385399182,11cyclictest0-21swapper/008:30:120
23853991810,5cyclictest15678-21sh12:30:140
2387799179,6cyclictest0-21swapper/310:32:263
2387799179,6cyclictest0-21swapper/309:52:543
2387799172,10cyclictest0-21swapper/309:00:133
2387799172,10cyclictest0-21swapper/308:50:043
2387799172,10cyclictest0-21swapper/307:20:123
2387799171,10cyclictest0-21swapper/307:55:133
2386199178,7cyclictest8462-21cut08:35:021
2386199175,8cyclictest21-21ksoftirqd/108:45:131
2386199175,8cyclictest21-21ksoftirqd/108:45:131
2386199172,11cyclictest23688-21kernelversion08:50:181
2386199171,8cyclictest17036-21sort08:10:151
2386199171,11cyclictest28157-21grep08:55:181
2386199171,11cyclictest28157-21grep08:55:181
2385399175,9cyclictest17221-21grep08:40:280
2385399175,9cyclictest17221-21grep08:40:280
2385399172,11cyclictest21880-21kthreadcore07:40:210
2385399172,10cyclictest0-21swapper/007:55:040
2385399171,13cyclictest24047-21ssh11:05:130
2385399171,13cyclictest22716-21apt-get08:50:010
2385399171,13cyclictest22716-21apt-get08:50:010
2385399171,11cyclictest383-21cut09:00:150
2387799161,9cyclictest0-21swapper/307:15:263
2387799161,9cyclictest0-21swapper/307:15:263
2387799161,12cyclictest0-21swapper/307:50:023
23877991610,4cyclictest0-21swapper/311:56:093
2386199161,11cyclictest24173-21grep07:10:161
2385399168,6cyclictest2732-21grep08:25:290
2385399168,5cyclictest10527-21irqrtprio09:10:180
2385399163,10cyclictest26283-21awk08:20:010
2385399163,10cyclictest19354-21kthreadcore08:10:200
2385399161,12cyclictest30335-21irqcore07:50:170
2385399161,12cyclictest15569-21ntp_states08:05:260
2385399161,10cyclictest839-21munin-node11:40:150
2387799151,10cyclictest0-21swapper/308:10:193
2385399151,10cyclictest23926-21latency_hist07:10:020
2387799149,3cyclictest0-21swapper/308:40:293
2387799149,3cyclictest0-21swapper/308:40:293
2387799141,7cyclictest0-21swapper/308:30:153
2387799138,3cyclictest0-21swapper/308:26:393
2387799131,9cyclictest0-21swapper/308:58:483
2387799131,9cyclictest0-21swapper/308:58:483
2387799121,9cyclictest0-21swapper/307:55:023
2387799121,7cyclictest0-21swapper/308:35:143
2387799112,7cyclictest0-21swapper/308:45:133
2387799112,7cyclictest0-21swapper/308:45:133
2387799111,7cyclictest0-21swapper/307:10:123
2387799101,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional