You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-09 - 07:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Fri May 09, 2025 00:41:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1595908994636,6cyclictest1683613-21munin-plugin-st22:25:012
1595907994637,5cyclictest1726150-21cpuspeed23:27:191
1595908994434,6cyclictest1640558-21gltestperf21:22:192
1595907994335,5cyclictest1673347-21ssh22:10:011
1595907994335,5cyclictest0-21swapper/121:40:011
1595907994334,6cyclictest1754140-21mailstats00:07:321
1595907994334,6cyclictest1-21systemd00:00:021
1595907994232,6cyclictest1741914-21systemd-run23:50:011
1595908994132,6cyclictest1748868-21cups-browsed00:32:212
1595908994132,5cyclictest1731651-21systemd-run23:35:022
1595908994130,8cyclictest266-21systemd-journal22:08:292
1595907994133,5cyclictest1649417-21grep21:35:011
1595907994133,5cyclictest1649417-21grep21:35:011
1595907994132,6cyclictest1668699-21threads22:02:371
1595908994033,3cyclictest0-21swapper/200:00:012
1595908994032,5cyclictest4093548-21cups-browsed21:28:542
1595908994030,7cyclictest1693862-21ssh22:40:022
1595907994032,5cyclictest1666491-21idleruntime-cro22:00:011
1595907994032,5cyclictest1-21systemd19:35:011
1595907994032,5cyclictest1-21systemd19:35:011
1595907994029,8cyclictest1719597-21ntpq23:17:211
1595908993931,5cyclictest1664518-21fschecks_count21:57:212
1595908993931,5cyclictest1630369-21idleruntime-cro21:05:012
1595908993931,5cyclictest1616939-21latency_hist20:20:022
1595908993930,6cyclictest1728179-21cron23:30:012
1595907993931,5cyclictest1635572-21munin-plugin-st21:15:011
1595907993931,5cyclictest1616888-21munin-run20:20:011
1595907993930,6cyclictest0-21swapper/121:50:011
1595907993929,7cyclictest4093548-21cups-browsed23:12:231
1595906993933,3cyclictest0-21swapper/000:00:010
1595908993830,5cyclictest1-21systemd23:05:012
1595908993828,8cyclictest422-21dbus-daemon23:20:022
1595907993829,5cyclictest491-21gdbus23:02:181
1595906993829,6cyclictest1-21systemd20:45:020
1595906993827,8cyclictest1758368-21sh00:15:010
1595906993824,11cyclictest0-21swapper/019:30:010
1595906993824,11cyclictest0-21swapper/019:30:010
1595908993729,5cyclictest1696282-21vmstat22:42:392
1595908993728,6cyclictest422-21dbus-daemon22:35:012
1595908993728,6cyclictest1626393-21fschecks_count20:52:142
1595908993728,6cyclictest1-21systemd21:00:012
1595906993731,3cyclictest0-21swapper/022:40:020
1595906993730,5cyclictest422-21dbus-daemon22:01:390
1595906993728,7cyclictest1642553-21/usr/sbin/munin21:25:020
1595906993725,9cyclictest1674996-21missed_timers22:12:220
1595908993630,3cyclictest0-21swapper/223:37:292
1595908993628,5cyclictest453-21systemd-logind19:40:012
1595908993628,5cyclictest453-21systemd-logind19:40:012
1595907993627,6cyclictest421-21avahi-daemon23:37:581
1595906993630,3cyclictest0-21swapper/022:32:140
1595906993628,5cyclictest1748868-21cups-browsed00:07:280
1595908993527,5cyclictest1651854-21unixbench_singl21:37:362
1595908993527,5cyclictest1630904-21grep21:07:182
1595908993526,7cyclictest1633391-21aten_r2power_vo21:12:162
1595908993526,6cyclictest1705200-21basename22:57:142
1595907993526,6cyclictest1603910-21fschecks_time19:37:181
1595907993526,6cyclictest1603910-21fschecks_time19:37:181
159590799351,26cyclictest1661561-21ssh21:52:291
1595906993529,4cyclictest1773894-21systemd-run00:40:010
1595906993528,4cyclictest1635731-21latency_hist21:15:030
1595906993527,5cyclictest422-21dbus-daemon23:20:590
1595906993525,8cyclictest1717802-21cron23:15:010
1595908993427,3cyclictest0-21swapper/200:12:302
1595908993426,5cyclictest4093548-21cups-browsed21:15:522
1595908993426,5cyclictest1-21systemd23:45:022
1595908993426,5cyclictest1-21systemd23:45:022
1595908993425,6cyclictest1754414-21unixbench-2d00:07:352
1595908993424,7cyclictest1749169-21sshd00:00:292
1595908993424,7cyclictest1668082-21latency22:02:222
1595908993327,3cyclictest0-21swapper/200:20:022
1595908993327,3cyclictest0-21swapper/200:20:022
1595908993325,5cyclictest1766699-21ssh00:27:262
1595908993325,5cyclictest1766699-21ssh00:27:262
1595908993325,5cyclictest1-21systemd22:27:352
1595908993324,4cyclictest0-21swapper/200:22:312
1595907993325,5cyclictest4093548-21cups-browsed22:35:111
1595907993324,6cyclictest421-21avahi-daemon22:57:261
1595907993324,5cyclictest453-21systemd-logind00:30:151
1595907993324,5cyclictest1745325-21cron23:55:011
1595907993319,8cyclictest1631858-21diskmemload23:22:331
1595906993328,2cyclictest0-21swapper/019:20:010
1595906993327,4cyclictest1681619-21ssh22:22:170
1595906993326,4cyclictest1665561-21head21:57:400
1595906993326,4cyclictest1644943-21grep21:27:410
1595906993325,6cyclictest1603376-21cut19:35:020
1595906993325,6cyclictest1603376-21cut19:35:020
1595906993325,5cyclictest266-21systemd-journal00:15:300
1595906993325,5cyclictest266-21systemd-journal00:15:300
1595906993323,7cyclictest0-21swapper/023:20:030
1595908993225,3cyclictest0-21swapper/221:41:292
1595908993224,5cyclictest1609353-21munin-plugin-st19:55:012
1595908993224,4cyclictest955-21mta-sts-daemon19:47:232
1595908993223,6cyclictest421-21avahi-daemon22:54:322
1595908993223,6cyclictest421-21avahi-daemon22:54:322
1595908993222,7cyclictest421-21avahi-daemon22:45:562
159590799323,23cyclictest1682039-21ssh22:22:241
1595907993224,4cyclictest1760588-21timerwakeupswit00:17:401
1595907993224,4cyclictest1760588-21timerwakeupswit00:17:401
1595907993223,7cyclictest1643824-21aten_r2power_po21:27:201
1595907993223,6cyclictest1610862-21sendmail20:00:011
1595907993223,5cyclictest0-21swapper/122:17:311
1595907993219,4cyclictest1631858-21diskmemload21:22:221
1595906993226,4cyclictest1700698-21latency_hist22:50:030
1595906993226,4cyclictest1696288-21ssh22:42:400
1595906993226,4cyclictest1659734-21systemd21:50:020
1595906993226,4cyclictest1638082-21unixbench_singl21:17:370
1595906993224,5cyclictest957-21mta-sts-daemon20:52:250
159590899318,7cyclictest59950lldpd23:06:212
1595908993126,2cyclictest0-21swapper/221:47:292
1595908993125,2cyclictest0-21swapper/221:32:322
1595908993125,2cyclictest0-21swapper/221:32:322
1595908993122,7cyclictest1598890-21sendmail19:20:012
1595908993119,9cyclictest422-21dbus-daemon21:52:232
1595907993125,3cyclictest0-21swapper/121:20:011
1595907993125,3cyclictest0-21swapper/119:15:011
1595907993125,2cyclictest0-21swapper/119:30:021
1595907993125,2cyclictest0-21swapper/119:30:021
1595907993123,6cyclictest1688065-21/usr/sbin/munin22:32:161
1595907993122,5cyclictest1758402-21systemd-run00:15:021
1595907993121,7cyclictest1700617-21idleruntime-cro22:50:011
1595907993120,7cyclictest422-21dbus-daemon00:35:421
1595907993118,10cyclictest1676767-21cron22:15:011
1595906993126,2cyclictest0-21swapper/022:27:390
1595906993125,4cyclictest1705208-21aten_r2power_fr22:57:150
1595906993125,4cyclictest1629725-21ntp_kernel_err21:02:230
1595906993125,4cyclictest1608767-21ntp_kernel_pll_19:52:230
1595906993125,3cyclictest1711347-21kworker/u8:16-xprtiod23:42:260
1595906993125,3cyclictest1711347-21kworker/u8:16-xprtiod23:42:260
1595906993124,5cyclictest422-21dbus-daemon23:30:020
1595906993124,4cyclictest1647263-21aten_r2power_vo21:32:150
1595906993124,4cyclictest1647263-21aten_r2power_vo21:32:150
1595906993123,6cyclictest1595866-21/usr/sbin/munin19:10:020
1595906993123,5cyclictest1748868-21cups-browsed00:28:210
1595906993123,5cyclictest1748868-21cups-browsed00:28:210
1595906993123,5cyclictest1748868-21cups-browsed00:00:150
1595906993122,6cyclictest1709054-21latency23:02:240
159590899304,12cyclictest1631858-21diskmemload23:53:072
1595908993024,3cyclictest0-21swapper/220:32:282
1595908993024,3cyclictest0-21swapper/200:40:012
1595908993022,5cyclictest1615343-21grep20:15:012
1595908993021,7cyclictest1625688-21timerwakeupswit20:47:362
1595907993024,3cyclictest0-21swapper/123:07:191
1595907993022,5cyclictest1768081-21systemd-run00:30:011
1595907993022,5cyclictest1768081-21systemd-run00:30:011
1595907993022,5cyclictest1621378-21grep20:35:011
1595907993022,5cyclictest1613853-21idleruntime-cro20:10:011
1595907993021,6cyclictest421-21avahi-daemon20:12:321
1595907993021,6cyclictest1625913-21latency_hist20:50:021
1595906993024,4cyclictest1672506-21sh22:07:570
1595906993024,4cyclictest1625752-21tail20:47:360
1595906993024,3cyclictest1743053-21aten2.4-expect23:52:120
1595906993024,3cyclictest1631858-21diskmemload21:42:360
1595908992923,3cyclictest0-21swapper/222:20:012
1595908992923,3cyclictest0-21swapper/222:12:252
1595908992922,5cyclictest1617139-21/usr/sbin/munin20:22:202
1595908992922,4cyclictest1596431-21fschecks_time19:12:162
1595907992923,3cyclictest0-21swapper/122:52:241
1595907992923,3cyclictest0-21swapper/122:52:241
1595907992921,5cyclictest957-21mta-sts-daemon19:52:231
1595907992921,5cyclictest453-21systemd-logind00:05:021
1595907992921,5cyclictest1697193-21cron22:45:011
1595906992923,4cyclictest421-21avahi-daemon19:37:320
1595906992923,4cyclictest421-21avahi-daemon19:37:320
1595906992923,3cyclictest0-21swapper/000:30:150
1595906992923,3cyclictest0-21swapper/000:22:160
1595906992919,7cyclictest1713396-21head23:07:390
1595906992918,9cyclictest0-21swapper/023:47:160
159590699291,23cyclictest1729982-21ssh23:32:270
159590899289,3cyclictest171rcu_preempt23:47:302
1595908992822,3cyclictest0-21swapper/219:22:262
1595908992821,5cyclictest0-21swapper/220:40:012
1595908992820,5cyclictest1715656-21aten2.4-expect23:12:152
1595908992818,7cyclictest1610852-21idleruntime-cro20:00:012
1595907992822,3cyclictest0-21swapper/119:10:021
1595907992821,4cyclictest1762871-21ssh00:22:161
1595907992821,3cyclictest0-21swapper/120:57:321
1595907992820,5cyclictest1606335-21unixbench_singl19:42:331
1595907992819,6cyclictest1630385-21systemd-run21:05:011
1595907992819,6cyclictest1598887-21grep19:20:011
1595907992819,5cyclictest0-21swapper/123:30:441
1595906992821,4cyclictest1650961-21kernelversion21:37:200
159590899275,5cyclictest1631858-21diskmemload23:21:162
1595908992721,3cyclictest0-21swapper/219:30:012
1595908992721,3cyclictest0-21swapper/219:30:012
1595908992719,5cyclictest4093548-21cups-browsed20:26:462
1595907992722,2cyclictest0-21swapper/121:45:011
1595907992722,2cyclictest0-21swapper/121:10:011
1595907992720,4cyclictest955-21mta-sts-daemon22:26:221
1595907992720,4cyclictest453-21systemd-logind19:50:011
1595907992720,3cyclictest0-21swapper/120:55:011
1595907992719,5cyclictest1623414-21sort20:42:151
1595907992719,5cyclictest1600335-21unixbench_singl19:22:311
159590699272,7cyclictest1733515-21ntpq23:37:290
1595906992723,2cyclictest0-21swapper/020:25:010
1595906992722,2cyclictest0-21swapper/022:52:190
1595906992722,2cyclictest0-21swapper/022:52:190
1595906992722,2cyclictest0-21swapper/021:52:410
1595906992721,3cyclictest0-21swapper/021:10:020
1595906992719,5cyclictest1607353-21kworker/u8:12+xprtiod20:40:010
1595907992618,5cyclictest1617444-21fschecks_time20:22:141
1595906992620,4cyclictest1606608-21/usr/sbin/munin19:47:230
1595906992620,4cyclictest1597400-21sh19:15:010
1595906992619,3cyclictest0-21swapper/022:16:030
1595908992518,3cyclictest0-21swapper/220:02:222
1595907992517,6cyclictest1612054-21switchtime20:02:321
1595906992520,3cyclictest1627489-21kworker/u8:10-xprtiod20:57:160
1595906992519,4cyclictest1611742-21collect220:02:210
1595906992515,7cyclictest1606371-21latency_hist19:45:000
1595906992511,12cyclictest1616455-21systemctl20:17:310
1595908992419,3cyclictest1595905-21cyclictest20:42:332
1595908992419,2cyclictest0-21swapper/219:45:012
1595908992414,7cyclictest1603395-21systemd-run19:35:012
1595908992414,7cyclictest1603395-21systemd-run19:35:012
1595907992418,3cyclictest0-21swapper/123:41:561
1595907992418,3cyclictest0-21swapper/123:41:561
159590799241,18cyclictest422-21dbus-daemon20:27:251
1595906992419,2cyclictest0-21swapper/020:27:350
1595906992419,2cyclictest0-21swapper/019:25:010
1595906992415,6cyclictest0-21swapper/020:15:020
159590799232,15cyclictest1622350-21ntp_states20:37:271
1595906992217,2cyclictest0-21swapper/020:35:010
1595906992217,2cyclictest0-21swapper/020:07:350
1595906992117,2cyclictest0-21swapper/019:55:290
159590899201,15cyclictest1613385-21ps20:07:302
1595908991810,5cyclictest1595911-21systemd-run19:10:012
159590999172,9cyclictest0-21swapper/321:07:263
1595909991711,4cyclictest0-21swapper/300:17:283
1595909991711,4cyclictest0-21swapper/300:17:283
159590999169,5cyclictest0-21swapper/300:27:293
159590999169,5cyclictest0-21swapper/300:27:293
159590999169,4cyclictest0-21swapper/300:07:303
1595909991610,4cyclictest0-21swapper/323:12:243
159590999160,13cyclictest0-21swapper/319:10:013
159590999159,4cyclictest0-21swapper/323:47:233
159590999159,4cyclictest0-21swapper/323:02:283
159590999159,4cyclictest0-21swapper/322:33:293
159590999152,10cyclictest0-21swapper/322:04:303
159590999151,10cyclictest0-21swapper/323:57:313
159590999151,10cyclictest0-21swapper/322:57:313
159590999151,10cyclictest0-21swapper/322:37:263
159590999151,10cyclictest0-21swapper/322:22:233
159590999151,10cyclictest0-21swapper/322:17:323
159590999151,10cyclictest0-21swapper/322:07:283
159590999151,10cyclictest0-21swapper/321:47:253
159590999151,10cyclictest0-21swapper/321:27:313
159590999151,10cyclictest0-21swapper/321:17:243
159590999151,10cyclictest0-21swapper/320:17:333
159590999151,10cyclictest0-21swapper/319:17:293
159590999151,10cyclictest0-21swapper/300:35:483
159590999151,10cyclictest0-21swapper/300:02:323
159590999150,11cyclictest0-21swapper/319:22:203
159590999150,10cyclictest0-21swapper/319:57:213
159590999148,4cyclictest0-21swapper/323:07:183
159590999148,4cyclictest0-21swapper/322:52:233
159590999148,4cyclictest0-21swapper/322:52:233
159590999143,8cyclictest0-21swapper/323:17:123
159590999143,8cyclictest0-21swapper/300:22:303
159590999142,9cyclictest0-21swapper/300:15:013
159590999141,9cyclictest0-21swapper/323:50:483
159590999141,9cyclictest0-21swapper/323:26:573
159590999141,9cyclictest0-21swapper/322:46:203
159590999141,9cyclictest0-21swapper/322:41:593
159590999141,9cyclictest0-21swapper/319:27:253
159590999141,9cyclictest0-21swapper/319:27:253
159590999141,9cyclictest0-21swapper/319:12:253
159590999141,10cyclictest0-21swapper/323:42:263
159590999141,10cyclictest0-21swapper/323:42:263
159590999141,10cyclictest0-21swapper/323:37:153
159590999141,10cyclictest0-21swapper/323:22:333
159590999141,10cyclictest0-21swapper/321:52:283
159590999141,10cyclictest0-21swapper/321:37:253
159590999141,10cyclictest0-21swapper/321:22:343
159590999141,10cyclictest0-21swapper/320:42:203
159590999141,10cyclictest0-21swapper/320:12:213
159590999141,10cyclictest0-21swapper/320:02:343
159590999141,10cyclictest0-21swapper/319:47:263
159590999141,10cyclictest0-21swapper/319:37:283
159590999141,10cyclictest0-21swapper/319:37:283
159590999141,10cyclictest0-21swapper/319:35:013
159590999141,10cyclictest0-21swapper/319:35:013
159590999140,10cyclictest0-21swapper/323:32:173
159590999140,10cyclictest0-21swapper/321:57:303
159590999140,10cyclictest0-21swapper/321:41:033
159590999140,10cyclictest0-21swapper/321:12:273
159590999140,10cyclictest0-21swapper/320:47:283
159590999140,10cyclictest0-21swapper/320:22:193
159590999140,10cyclictest0-21swapper/320:07:293
159590999131,9cyclictest0-21swapper/322:26:373
159590999131,9cyclictest0-21swapper/322:11:263
159590999131,9cyclictest0-21swapper/321:02:143
159590999131,9cyclictest0-21swapper/320:57:163
159590999131,9cyclictest0-21swapper/320:52:263
159590999131,9cyclictest0-21swapper/320:37:263
159590999131,9cyclictest0-21swapper/320:32:243
159590999131,9cyclictest0-21swapper/320:30:013
159590999131,9cyclictest0-21swapper/319:50:283
159590999131,9cyclictest0-21swapper/319:42:173
159590999131,9cyclictest0-21swapper/300:30:163
159590999131,8cyclictest0-21swapper/321:30:033
159590999131,8cyclictest0-21swapper/321:30:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional