You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-31 - 05:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Sun Aug 31, 2025 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
58799260,2cyclictest26903-21awk00:20:001
61199250,24cyclictest17826-21apt-get22:55:023
61199240,23cyclictest9556-21cron21:34:593
58799240,1cyclictest30174-21apt-get21:10:021
581992423,1cyclictest0-21swapper/023:04:550
581992422,1cyclictest9282-21cron20:14:590
58199240,23cyclictest0-21swapper/023:39:580
61199231,1cyclictest27603-21idleruntime-cro21:04:593
61199230,22cyclictest6951-21perf23:40:003
61199230,22cyclictest24727-21sed22:05:013
61199230,22cyclictest1641-21cron22:10:013
60499230,22cyclictest4515-21munin-run23:35:002
60499230,22cyclictest17114-21idleruntime-cro22:55:012
587992321,1cyclictest211rcuc/122:24:591
587992320,2cyclictest4480-21cron22:29:591
58799230,22cyclictest0-21swapper/122:49:551
58799230,0cyclictest23446-21apt-get00:14:571
581992321,1cyclictest0-21swapper/021:25:000
611992221,1cyclictest28808-21apt-get22:14:583
611992220,1cyclictest30905-21sed21:10:013
611992220,1cyclictest26934-21perf00:20:003
611992220,1cyclictest11956-21apt20:35:013
61199220,21cyclictest13668-21apt-get22:49:533
61199220,21cyclictest1252-21apt-get22:24:543
61199220,21cyclictest0-21swapper/323:44:563
604992221,1cyclictest19075-21apt-get20:49:562
604992220,1cyclictest6947-21munin-plugin-st23:40:002
604992220,1cyclictest2934-21perf21:20:012
604992220,1cyclictest0-21swapper/219:50:002
60499220,21cyclictest6744-21apt-get21:29:592
60499220,21cyclictest4500-21apt-get22:30:012
587992220,1cyclictest5683-21basename21:25:001
58799220,2cyclictest4516-21grep23:35:001
58799220,21cyclictest3791-21awk19:35:001
58799220,21cyclictest0-21swapper/122:54:561
58799220,21cyclictest0-21swapper/121:09:561
58799220,1cyclictest0-21swapper/123:14:591
581992221,1cyclictest0-21swapper/020:04:590
581992220,1cyclictest4502-21perf23:35:000
581992220,1cyclictest30409-21apt-config21:10:000
581992220,1cyclictest22563-21apt-get20:55:020
58199220,2cyclictest6737-21dump-pmu-power19:55:000
58199220,21cyclictest9458-21memory23:45:010
58199220,21cyclictest13958-21apt-get21:44:570
58199220,1cyclictest593-21dbus-daemon22:05:000
58199220,1cyclictest0-21swapper/022:54:560
611992121,0cyclictest0-21swapper/323:55:013
611992121,0cyclictest0-21swapper/320:59:553
611992120,0cyclictest5343-21apt-get22:30:013
611992119,1cyclictest4500-21apt22:30:003
611992119,1cyclictest20302-21memory21:55:013
611992119,1cyclictest15521-21perf20:40:003
611992118,2cyclictest22105-21basename23:04:593
61199210,20cyclictest3043-21apt-get00:39:573
61199210,20cyclictest0-21swapper/323:14:593
61199210,1cyclictest0-21swapper/300:30:013
61199210,0cyclictest0-21swapper/321:24:543
604992120,1cyclictest26411-21apt-get22:09:562
604992119,1cyclictest9464-21cron23:45:002
604992119,1cyclictest1588-21cron00:34:592
60499211,19cyclictest29564-21cron23:20:012
60499210,20cyclictest19465-21apt-get21:55:022
60499210,20cyclictest0-21swapper/220:15:002
60499210,1cyclictest8010-21idleruntime-cro20:05:002
60499210,1cyclictest26735-21apt-get21:04:572
60499210,1cyclictest15501-21turbostat20:40:002
60499210,0cyclictest0-21swapper/220:54:592
587992121,0cyclictest30937-21apt-get23:24:521
587992120,1cyclictest19497-21perf00:05:011
587992119,1cyclictest5346-21ntpq22:30:011
58799210,21cyclictest0-21swapper/100:09:551
58799210,20cyclictest925-21apt-get23:29:551
58799210,1cyclictest15008-21ntpq23:55:011
58799210,0cyclictest16352-21apt-get21:49:561
58799210,0cyclictest0-21swapper/123:39:571
58799210,0cyclictest0-21swapper/121:34:581
58799210,0cyclictest0-21swapper/120:55:001
581992121,0cyclictest0-21swapper/021:34:590
581992121,0cyclictest0-21swapper/021:14:550
581992121,0cyclictest0-21swapper/000:04:560
581992120,1cyclictest10952-21apt-get23:49:570
58199210,20cyclictest0-21swapper/000:39:560
58199210,1cyclictest32339-21apt23:25:000
58199210,1cyclictest22117-21awk00:10:000
58199210,0cyclictest31205-21apt-get22:19:560
58199210,0cyclictest0-21swapper/022:49:550
58199210,0cyclictest0-21swapper/000:34:550
58199210,0cyclictest0-21swapper/000:14:560
611992020,0cyclictest0-21swapper/323:29:563
611992020,0cyclictest0-21swapper/323:09:573
611992020,0cyclictest0-21swapper/321:30:003
611992020,0cyclictest0-21swapper/320:49:563
611992020,0cyclictest0-21swapper/320:44:523
611992019,1cyclictest0-21swapper/321:54:573
611992019,0cyclictest11281-21apt-get22:44:573
611992019,0cyclictest0-21swapper/323:19:563
611992018,1cyclictest32041-21perf22:20:003
61199200,0cyclictest0-21swapper/321:44:573
61199200,0cyclictest0-21swapper/321:19:573
61199200,0cyclictest0-21swapper/319:50:013
604992020,0cyclictest0-21swapper/223:49:562
604992020,0cyclictest0-21swapper/221:54:582
604992020,0cyclictest0-21swapper/221:34:582
604992020,0cyclictest0-21swapper/200:04:562
604992019,1cyclictest26131-21apt-get23:14:592
604992019,1cyclictest13348-21apt-get23:54:552
604992017,2cyclictest7400-21basename20:00:012
604992017,2cyclictest12667-21perf21:39:592
60499200,20cyclictest0-21swapper/222:39:562
60499200,1cyclictest3862-21perf00:39:592
60499200,19cyclictest0-21swapper/222:50:002
60499200,0cyclictest13958-21apt-get21:44:572
60499200,0cyclictest0-21swapper/223:55:012
60499200,0cyclictest0-21swapper/222:24:532
60499200,0cyclictest0-21swapper/220:34:552
587992020,0cyclictest0-21swapper/123:04:561
587992020,0cyclictest0-21swapper/121:44:581
587992019,0cyclictest0-21swapper/122:04:571
587992018,1cyclictest29646-21munin-run22:15:001
58799200,20cyclictest0-21swapper/123:19:551
58799200,0cyclictest0-21swapper/122:44:581
581992020,0cyclictest0-21swapper/022:29:570
581992020,0cyclictest0-21swapper/022:24:540
581992020,0cyclictest0-21swapper/000:29:580
581992020,0cyclictest0-21swapper/000:24:580
581992018,1cyclictest612-21gdbus19:35:000
581992018,1cyclictest21509-21apt-get20:55:000
581992018,1cyclictest0-21swapper/021:55:000
581992018,1cyclictest0-21swapper/020:30:000
58199200,0cyclictest0-21swapper/022:44:580
611991919,0cyclictest29130-21apt-get21:09:553
611991919,0cyclictest21047-21apt-get00:09:553
611991918,0cyclictest0-21swapper/300:34:563
611991917,1cyclictest2678-21apt-get23:30:023
61199190,18cyclictest593-21dbus-daemon00:25:013
604991919,0cyclictest23960-21apt-get22:04:582
604991919,0cyclictest0-21swapper/222:44:572
604991919,0cyclictest0-21swapper/222:14:572
604991919,0cyclictest0-21swapper/221:49:572
604991919,0cyclictest0-21swapper/200:09:562
604991918,0cyclictest0-21swapper/219:50:012
604991916,2cyclictest3768-21cron19:35:002
587991919,0cyclictest0-21swapper/121:59:551
587991919,0cyclictest0-21swapper/121:29:591
587991919,0cyclictest0-21swapper/121:05:001
587991917,1cyclictest9286-21awk20:14:591
587991917,1cyclictest5083-21wc19:45:011
587991917,1cyclictest5083-21wc19:45:001
587991917,1cyclictest0-21swapper/121:40:001
581991919,0cyclictest0-21swapper/023:54:560
581991919,0cyclictest0-21swapper/023:29:550
581991919,0cyclictest0-21swapper/020:49:570
581991919,0cyclictest0-21swapper/020:34:570
581991918,1cyclictest0-21swapper/021:40:000
581991918,0cyclictest23679-21apt-get23:09:560
611991818,0cyclictest11535-21apt-get21:39:533
611991818,0cyclictest0-21swapper/320:54:593
604991818,0cyclictest0-21swapper/221:14:562
604991818,0cyclictest0-21swapper/220:44:512
604991818,0cyclictest0-21swapper/200:19:562
604991817,1cyclictest0-21swapper/221:09:562
587991818,0cyclictest0-21swapper/120:49:561
587991818,0cyclictest0-21swapper/120:39:511
587991817,1cyclictest0-21swapper/123:49:571
587991817,1cyclictest0-21swapper/119:24:591
587991816,1cyclictest31255-21cron00:30:001
581991818,0cyclictest0-21swapper/023:19:550
581991818,0cyclictest0-21swapper/022:39:570
581991818,0cyclictest0-21swapper/021:29:580
581991816,1cyclictest4433-21sendmail-msp19:40:010
581991816,1cyclictest1214-21awk19:15:000
58199180,0cyclictest0-21swapper/021:49:560
611991717,0cyclictest18655-21apt-get00:04:553
61199170,16cyclictest15815-21sed21:45:053
604991717,0cyclictest0-21swapper/223:24:522
604991717,0cyclictest0-21swapper/223:09:572
604991716,1cyclictest24383-21munin-run00:15:012
587991717,0cyclictest9499-21perf23:45:001
587991717,0cyclictest0-21swapper/120:44:521
587991714,2cyclictest10581-21cron20:25:001
58799170,0cyclictest0-21swapper/100:34:551
581991717,0cyclictest0-21swapper/022:14:560
581991715,1cyclictest7363-21memory22:35:010
581991715,1cyclictest2935-21sendmail21:20:000
611991616,0cyclictest0-21swapper/323:54:553
611991616,0cyclictest0-21swapper/323:24:533
611991615,1cyclictest17113-21apt-get22:55:003
61199160,15cyclictest1349-21munin-run19:14:593
604991616,0cyclictest0-21swapper/223:29:552
604991616,0cyclictest0-21swapper/220:59:552
587991616,0cyclictest0-21swapper/122:59:571
587991616,0cyclictest0-21swapper/100:24:581
581991616,0cyclictest26131-21apt-get23:14:580
581991615,1cyclictest0-21swapper/020:09:590
58199160,15cyclictest0-21swapper/019:29:590
60499154,8cyclictest0-21swapper/221:20:012
604991515,0cyclictest0-21swapper/222:55:032
604991514,1cyclictest21281-21apt-get23:04:572
604991514,1cyclictest11220-21turbostat20:30:002
58799153,8cyclictest211rcuc/121:55:011
587991515,0cyclictest0-21swapper/122:19:561
587991515,0cyclictest0-21swapper/122:09:561
587991513,1cyclictest14294-21perf23:55:001
58799150,1cyclictest1948-21ntp_states00:35:011
581991515,0cyclictest0-21swapper/021:59:550
611991414,0cyclictest0-21swapper/323:46:343
611991414,0cyclictest0-21swapper/320:25:413
611991414,0cyclictest0-21swapper/319:57:003
611991414,0cyclictest0-21swapper/319:22:593
611991414,0cyclictest0-21swapper/300:10:443
61199140,14cyclictest0-21swapper/322:35:543
61199140,14cyclictest0-21swapper/320:22:493
61199140,14cyclictest0-21swapper/320:16:483
61199140,14cyclictest0-21swapper/320:08:073
61199140,14cyclictest0-21swapper/319:49:523
61199140,14cyclictest0-21swapper/319:43:163
61199140,14cyclictest0-21swapper/319:43:163
61199140,14cyclictest0-21swapper/319:35:343
61199140,0cyclictest0-21swapper/320:13:103
61199140,0cyclictest0-21swapper/320:03:153
604991412,1cyclictest10583-21basename20:25:002
60499140,14cyclictest0-21swapper/220:09:072
60499140,14cyclictest0-21swapper/219:40:562
60499140,14cyclictest0-21swapper/219:40:562
60499140,14cyclictest0-21swapper/219:36:452
60499140,14cyclictest0-21swapper/219:15:102
60499140,14cyclictest0-21swapper/219:13:082
60499140,0cyclictest0-21swapper/222:25:312
60499140,0cyclictest0-21swapper/219:21:282
587991414,0cyclictest0-21swapper/122:35:581
587991414,0cyclictest0-21swapper/120:19:281
587991414,0cyclictest0-21swapper/119:50:201
58799140,14cyclictest0-21swapper/121:16:211
58799140,14cyclictest0-21swapper/120:25:261
58799140,14cyclictest0-21swapper/120:08:531
58799140,14cyclictest0-21swapper/119:55:281
58799140,14cyclictest0-21swapper/119:45:251
58799140,14cyclictest0-21swapper/119:36:461
58799140,14cyclictest0-21swapper/119:10:391
58799140,0cyclictest0-21swapper/123:07:151
58799140,0cyclictest0-21swapper/119:25:051
58799140,0cyclictest0-21swapper/119:15:281
581991414,0cyclictest0-21swapper/023:55:050
581991413,1cyclictest5191-21munin-node22:05:560
581991413,1cyclictest0-21swapper/022:55:010
58199140,14cyclictest0-21swapper/020:15:460
58199140,14cyclictest0-21swapper/019:57:050
58199140,14cyclictest0-21swapper/019:23:290
58199140,14cyclictest0-21swapper/019:17:380
58199140,0cyclictest0-21swapper/021:03:380
58199140,0cyclictest0-21swapper/020:42:120
58199140,0cyclictest0-21swapper/020:37:250
58199140,0cyclictest0-21swapper/019:45:220
58199140,0cyclictest0-21swapper/000:15:460
611991313,0cyclictest0-21swapper/319:31:053
611991313,0cyclictest0-21swapper/319:15:173
61199130,0cyclictest0-21swapper/319:25:373
604991313,0cyclictest0-21swapper/220:15:402
604991313,0cyclictest0-21swapper/219:28:462
604991313,0cyclictest0-21swapper/200:21:132
587991313,0cyclictest0-21swapper/120:32:211
581991313,0cyclictest0-21swapper/020:21:130
581991313,0cyclictest0-21swapper/019:40:010
581991313,0cyclictest0-21swapper/019:40:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional