You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-14 - 06:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Sun Dec 14, 2025 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13557992520,1cyclictest151rcu_preempt00:35:010
1356799240,23cyclictest31676-21grep22:00:003
1356799240,23cyclictest18501-21sendmail-msp22:40:003
1356199240,23cyclictest16074-21dump-pmu-power22:35:011
13557992420,3cyclictest12376-21cron21:20:010
13557992420,2cyclictest313-21systemd-journal22:55:000
1355799240,3cyclictest30540-21awk23:05:010
13567992320,2cyclictest3286-21cron23:15:013
1356799230,22cyclictest5702-21sh23:20:013
1356799230,1cyclictest0-21swapper/321:55:003
1356799230,1cyclictest0-21swapper/300:20:023
13563992321,1cyclictest7967-21awk20:05:012
13563992321,1cyclictest10945-21apt-get20:14:202
13561992321,1cyclictest5520-21cron20:00:011
13561992321,1cyclictest4654-21apt-get22:14:191
13561992321,1cyclictest18755-21apt-get23:45:021
1356199230,2cyclictest12761-21munin-run20:15:001
1356199230,22cyclictest25951-21cron19:35:001
1356199230,22cyclictest19977-21perf20:30:001
1355799230,22cyclictest7617-21perf21:10:000
1355799230,22cyclictest26874-21apt-get00:04:190
1355799230,0cyclictest0-21swapper/019:19:190
13567992220,1cyclictest26873-21apt00:04:193
1356799220,2cyclictest13884-21perf00:40:013
1356799220,21cyclictest8884-21perf22:20:013
1356799220,21cyclictest8884-21perf22:20:013
1356799220,21cyclictest0-21swapper/321:19:183
1356799220,21cyclictest0-21swapper/319:54:173
1356799220,1cyclictest11468-21latency_hist00:35:013
13563992221,0cyclictest12046-21apt-get00:39:192
13563992220,1cyclictest0-21swapper/219:25:002
1356399220,21cyclictest606-21dbus-daemon21:00:012
1356399220,21cyclictest26200-21sendmail-msp00:00:012
1356399220,21cyclictest0-21swapper/223:54:192
1356399220,21cyclictest0-21swapper/219:19:192
1356399220,1cyclictest8102-21cron23:25:002
1356399220,1cyclictest30764-21turbostat.cron19:45:012
13561992222,0cyclictest0-21swapper/121:49:191
13561992220,1cyclictest17559-21sh20:25:001
1356199220,21cyclictest29995-21cron20:50:011
1356199220,21cyclictest20883-21cron22:45:001
1356199220,21cyclictest2057-21apt-get00:19:201
1356199220,21cyclictest0-21swapper/123:34:181
1356199220,21cyclictest0-21swapper/123:04:171
1356199220,21cyclictest0-21swapper/119:45:011
1356199220,21cyclictest0-21swapper/119:19:191
1356199220,21cyclictest0-21swapper/100:10:031
1356199220,1cyclictest3876-21apt-get00:20:021
13557992221,1cyclictest0-21swapper/019:10:020
13557992220,1cyclictest9081-21munin-run00:30:010
13557992220,1cyclictest3739-21apt-get19:59:190
13557992219,2cyclictest25073-21taskset21:49:190
1355799220,21cyclictest0-21swapper/023:24:190
1355799220,21cyclictest0-21swapper/021:24:180
1355799220,21cyclictest0-21swapper/021:04:180
1355799220,21cyclictest0-21swapper/020:29:210
1355799220,1cyclictest3888-21munin-run00:20:000
1356799210,20cyclictest32064-21apt-get00:14:213
1356799210,20cyclictest21525-21apt-get22:49:193
1356799210,0cyclictest0-21swapper/322:04:193
1356799210,0cyclictest0-21swapper/320:54:183
13563992121,0cyclictest0-21swapper/223:04:182
13563992120,1cyclictest14260-21apt-get22:34:172
13563992120,1cyclictest0-21swapper/223:09:202
13563992120,0cyclictest0-21swapper/223:29:192
13563992119,1cyclictest23924-21apt-get22:54:202
13563992119,1cyclictest0-21swapper/219:54:172
1356399210,21cyclictest0-21swapper/200:19:202
1356399210,20cyclictest32275-21apt-get22:04:202
1356399210,1cyclictest29267-21apt-get00:09:192
1356399210,0cyclictest26547-21apt-get19:39:182
13561992120,1cyclictest14493-21apt-get19:14:181
13561992119,1cyclictest0-21swapper/123:55:021
13561992119,1cyclictest0-21swapper/120:35:011
1356199210,20cyclictest9656-21apt-get00:34:201
1356199210,20cyclictest0-21swapper/122:54:191
1356199210,20cyclictest0-21swapper/122:24:211
1356199210,1cyclictest1492-21apt-get23:14:171
1356199210,0cyclictest0-21swapper/120:19:181
13557992121,0cyclictest0-21swapper/022:44:180
13557992120,1cyclictest8193-21apt-get21:14:170
13557992119,1cyclictest13911-21apt-get23:39:200
13557992111,9cyclictest13-21ksoftirqd/021:00:010
1355799210,20cyclictest31357-21apt-get19:49:190
1355799210,20cyclictest0-21swapper/023:34:180
1355799210,20cyclictest0-21swapper/019:29:180
1355799210,1cyclictest0-21swapper/023:19:190
1355799210,0cyclictest0-21swapper/023:49:190
1355799210,0cyclictest0-21swapper/000:24:190
13567992020,0cyclictest0-21swapper/323:45:023
13567992020,0cyclictest0-21swapper/322:14:193
13567992020,0cyclictest0-21swapper/321:44:183
13567992020,0cyclictest0-21swapper/320:39:183
13567992018,1cyclictest606-21dbus-daemon20:45:013
13567992018,1cyclictest26201-21munin-run00:00:013
13567992018,1cyclictest12742-21grep20:15:003
1356799200,20cyclictest0-21swapper/322:59:163
1356799200,20cyclictest0-21swapper/320:34:193
1356799200,0cyclictest0-21swapper/323:54:193
1356799200,0cyclictest0-21swapper/322:54:203
13563992020,0cyclictest21755-21apt-get19:29:192
13563992020,0cyclictest0-21swapper/221:15:002
13563992020,0cyclictest0-21swapper/221:04:192
13563992020,0cyclictest0-21swapper/219:14:182
13563992020,0cyclictest0-21swapper/200:34:202
13563992018,1cyclictest606-21dbus-daemon20:35:012
13563992018,1cyclictest0-21swapper/200:14:212
1356399200,19cyclictest0-21swapper/219:59:192
1356399200,0cyclictest0-21swapper/222:09:182
1356399200,0cyclictest0-21swapper/221:19:182
1356399200,0cyclictest0-21swapper/219:34:192
13561992020,0cyclictest8545-21apt-get20:09:161
13561992020,0cyclictest0-21swapper/123:44:181
13561992020,0cyclictest0-21swapper/123:05:021
13561992020,0cyclictest0-21swapper/122:39:181
13561992020,0cyclictest0-21swapper/121:54:171
13561992020,0cyclictest0-21swapper/121:24:181
13561992020,0cyclictest0-21swapper/100:39:191
13561992019,1cyclictest3904-21apt-get23:19:191
1356199200,20cyclictest0-21swapper/122:19:191
1356199200,20cyclictest0-21swapper/122:19:191
1356199200,20cyclictest0-21swapper/121:44:181
1356199200,0cyclictest0-21swapper/122:09:181
1356199200,0cyclictest0-21swapper/121:39:161
1356199200,0cyclictest0-21swapper/120:39:181
13557992020,0cyclictest23005-21apt-get20:39:170
13557992020,0cyclictest11858-21apt-get22:29:180
13557992020,0cyclictest0-21swapper/023:59:200
13557992020,0cyclictest0-21swapper/022:09:170
13557992020,0cyclictest0-21swapper/021:29:180
13557992020,0cyclictest0-21swapper/020:09:160
13557992020,0cyclictest0-21swapper/000:14:210
13557992019,1cyclictest22672-21apt-get21:44:180
13557992019,1cyclictest13-21ksoftirqd/020:55:010
1355799200,19cyclictest0-21swapper/020:49:220
1355799200,0cyclictest29857-21apt-get21:59:190
13567991919,0cyclictest0-21swapper/323:38:053
13567991919,0cyclictest0-21swapper/319:44:183
13567991919,0cyclictest0-21swapper/319:34:183
13567991918,1cyclictest28214-21apt-get20:49:203
13567991918,1cyclictest0-21swapper/321:00:033
1356799190,1cyclictest30550-21munin-run23:05:013
1356799190,0cyclictest2259-21apt-get22:09:183
1356799190,0cyclictest0-21swapper/321:14:173
13563991919,0cyclictest15761-21apt-get20:24:172
13563991919,0cyclictest0-21swapper/223:30:022
13563991919,0cyclictest0-21swapper/222:19:182
13563991919,0cyclictest0-21swapper/222:19:182
13563991919,0cyclictest0-21swapper/221:59:182
13563991919,0cyclictest0-21swapper/200:20:022
13563991917,1cyclictest30607-21apt-get20:54:172
13561991919,0cyclictest7262-21apt-get00:29:191
13561991919,0cyclictest0-21swapper/123:39:201
13561991919,0cyclictest0-21swapper/122:29:181
13561991919,0cyclictest0-21swapper/120:59:201
13561991919,0cyclictest0-21swapper/120:44:181
13561991919,0cyclictest0-21swapper/119:29:191
13561991919,0cyclictest0-21swapper/100:09:191
13561991918,0cyclictest0-21swapper/122:04:201
13557991919,0cyclictest29267-21apt-get00:09:190
13557991919,0cyclictest0-21swapper/022:24:210
13557991919,0cyclictest0-21swapper/022:04:190
13557991919,0cyclictest0-21swapper/020:44:180
13557991919,0cyclictest0-21swapper/020:14:200
13567991818,0cyclictest0-21swapper/323:09:203
13567991816,1cyclictest21153-21ls19:25:003
13567991816,1cyclictest19076-21apt-get22:44:193
13567991815,2cyclictest19954-21idleruntime-cro20:30:003
1356799180,17cyclictest0-21swapper/323:44:183
13563991818,0cyclictest5799-21apt-get21:09:192
13563991818,0cyclictest27462-21apt-get21:54:172
13563991818,0cyclictest19119-21apt-get23:49:202
13563991818,0cyclictest0-21swapper/222:14:192
13563991818,0cyclictest0-21swapper/220:09:162
13563991818,0cyclictest0-21swapper/200:29:192
13563991817,0cyclictest13337-21apt-get20:19:182
1356399180,18cyclictest0-21swapper/223:19:192
13561991816,1cyclictest10529-21awk23:30:001
13557991818,0cyclictest0-21swapper/023:44:180
13557991818,0cyclictest0-21swapper/023:29:190
13557991818,0cyclictest0-21swapper/021:39:160
13557991817,1cyclictest6153-21apt-get20:04:180
13557991817,1cyclictest21571-21apt-get23:54:190
13567991717,0cyclictest8713-21apt-get23:29:193
13567991717,0cyclictest0-21swapper/323:34:183
13567991717,0cyclictest0-21swapper/319:39:183
13567991717,0cyclictest0-21swapper/300:29:203
13563991717,0cyclictest0-21swapper/222:49:192
13563991717,0cyclictest0-21swapper/220:45:012
13563991717,0cyclictest0-21swapper/219:49:192
13561991717,0cyclictest1329-21apt-get19:54:171
13561991717,0cyclictest0-21swapper/121:29:181
13561991717,0cyclictest0-21swapper/121:14:161
13561991716,1cyclictest0-21swapper/123:54:191
13557991717,0cyclictest0-21swapper/019:34:190
13557991714,2cyclictest13-21ksoftirqd/000:39:190
1355799170,16cyclictest31074-21sort23:05:010
13567991616,0cyclictest13007-21apt-get21:24:183
13567991616,0cyclictest0-21swapper/322:24:213
13567991616,0cyclictest0-21swapper/321:34:183
13567991616,0cyclictest0-21swapper/320:19:183
13567991614,1cyclictest0-21swapper/321:39:293
1356799160,1cyclictest0-21swapper/321:00:013
13563991616,0cyclictest0-21swapper/222:59:172
13563991616,0cyclictest0-21swapper/220:49:202
13563991615,1cyclictest0-21swapper/200:04:192
13563991615,0cyclictest16325-21apt-get23:44:182
13563991613,2cyclictest13677-21munin-run22:30:012
1356399160,16cyclictest0-21swapper/221:21:462
13561991616,0cyclictest0-21swapper/121:59:181
13561991614,1cyclictest8137-21idleruntime-cro23:25:001
1356199160,15cyclictest32404-21awk20:55:011
1356199160,15cyclictest0-21swapper/119:47:311
1355799165,9cyclictest13-21ksoftirqd/020:24:170
13557991616,0cyclictest16659-21apt-get22:39:190
1355799160,1cyclictest0-21swapper/022:14:260
1355799160,15cyclictest313-21systemd-journal22:20:010
1355799160,15cyclictest313-21systemd-journal22:20:010
1356799155,8cyclictest1824-21sendmail-mta19:13:173
13567991515,0cyclictest15409-21apt-get21:29:183
13567991515,0cyclictest0-21swapper/319:59:193
13567991515,0cyclictest0-21swapper/319:49:193
13567991515,0cyclictest0-21swapper/300:19:203
13567991514,1cyclictest0-21swapper/300:09:193
13567991513,1cyclictest0-21swapper/321:10:003
13563991515,0cyclictest0-21swapper/222:44:192
13563991515,0cyclictest0-21swapper/220:39:172
13563991515,0cyclictest0-21swapper/220:29:212
13561991513,1cyclictest0-21swapper/119:25:001
13557991515,0cyclictest0-21swapper/020:19:180
13557991513,1cyclictest20356-21ntp_states19:24:230
1355799150,15cyclictest1190-21taskset23:10:470
1355799150,10cyclictest783-21snmpd22:34:250
13567991414,0cyclictest0-21swapper/323:20:323
13567991414,0cyclictest0-21swapper/322:25:263
13567991414,0cyclictest0-21swapper/320:00:443
13567991414,0cyclictest0-21swapper/319:25:223
13567991414,0cyclictest0-21swapper/319:15:053
1356799140,14cyclictest0-21swapper/322:31:053
1356799140,14cyclictest0-21swapper/321:46:423
1356799140,14cyclictest0-21swapper/320:20:033
1356799140,14cyclictest0-21swapper/320:05:393
13563991414,0cyclictest0-21swapper/221:40:242
1356399140,14cyclictest0-21swapper/222:20:092
1356399140,14cyclictest0-21swapper/221:45:032
1356399140,14cyclictest0-21swapper/221:31:072
1356399140,0cyclictest0-21swapper/223:36:292
1356399140,0cyclictest0-21swapper/222:36:262
1356399140,0cyclictest0-21swapper/221:36:102
1356399140,0cyclictest0-21swapper/221:26:132
13561991414,0cyclictest0-21swapper/119:35:121
1356199140,14cyclictest0-21swapper/121:30:091
1356199140,14cyclictest0-21swapper/100:00:021
1356199140,0cyclictest0-21swapper/121:05:071
1356199140,0cyclictest0-21swapper/120:00:311
13557991414,0cyclictest0-21swapper/021:30:420
1355799140,14cyclictest0-21swapper/022:56:150
1355799140,14cyclictest0-21swapper/020:30:260
1355799140,14cyclictest0-21swapper/019:40:290
1355799140,14cyclictest0-21swapper/019:35:300
1355799140,0cyclictest0-21swapper/022:46:470
1355799140,0cyclictest0-21swapper/021:50:420
1355799140,0cyclictest0-21swapper/019:51:300
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional