You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-29 - 15:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Wed Oct 29, 2025 12:39:59)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1490478995949,6cyclictest1563412-21kworker/u8:10-events_unbound10:09:061
1490478995444,6cyclictest421-21avahi-daemon11:43:231
1490479995344,6cyclictest1499460-21munin-run07:40:012
1490479994942,4cyclictest1619287-21kworker/u8:6-nfsiod11:44:082
1490479994941,5cyclictest1596741-21systemd-run10:55:022
1490479994839,5cyclictest1536742-21latency_hist09:25:022
1490478994738,6cyclictest1570209-21true10:15:011
1490479994636,6cyclictest1533306-21idleruntime-cro09:20:012
1490478994636,6cyclictest1660342-21gltestperf12:33:541
1490478994537,3cyclictest0-21swapper/109:59:031
1490479994436,5cyclictest1500986-21systemd-run07:45:012
1490479994431,8cyclictest0-21swapper/210:58:502
1490477994422,15cyclictest151ktimers/008:09:030
1490479994335,5cyclictest1508547-21latency_hist08:10:022
1490479994334,5cyclictest1572733-21sshd10:19:062
1490479994332,7cyclictest1493515-21latency_hist07:20:022
1490478994335,5cyclictest1536702-21sh09:25:011
1490478994234,5cyclictest1505512-21systemctl08:00:011
1490478994234,4cyclictest1493505-21systemd-run07:20:011
1490478994233,6cyclictest1-21systemd11:00:021
1490477994216,21cyclictest1579295-21ssh10:29:020
1490479994116,16cyclictest1559116-21sh09:59:042
1490478994133,5cyclictest1654736-21latency_hist12:25:021
1490477994135,4cyclictest1603049-21grep11:05:010
1490479994032,4cyclictest0-21swapper/211:03:512
1490479994029,7cyclictest266-21systemd-journal11:17:052
1490478994035,2cyclictest0-21swapper/110:34:061
1490478994033,4cyclictest1586785-21idleruntime-cro10:40:011
1490478994032,5cyclictest959-21mta-sts-daemon09:04:061
1490478994032,5cyclictest959-21mta-sts-daemon09:04:061
149047799405,29cyclictest1595870-21ssh10:54:050
1490479993926,8cyclictest422-21dbus-daemon12:39:012
149047999392,30cyclictest1634368-21ssh11:54:002
1490479993910,12cyclictest1555998-21sh09:54:072
1490478993931,5cyclictest453-21systemd-logind10:25:021
1490478993928,9cyclictest453-21systemd-logind09:55:021
1490478993924,12cyclictest1572930-21sh10:19:121
1490477993931,5cyclictest266-21systemd-journal09:59:050
1490479993829,5cyclictest1525038-21systemd-run09:05:012
1490479993829,5cyclictest1525038-21systemd-run09:05:012
1490479993828,6cyclictest0-21swapper/212:09:032
1490479993812,13cyclictest1497586-21run-parts07:34:092
1490478993829,4cyclictest0-21swapper/112:09:001
1490477993832,4cyclictest1510005-21latency_hist08:15:010
1490477993832,4cyclictest1510005-21latency_hist08:15:010
1490477993831,4cyclictest1586791-21ls10:40:010
149047999373,25cyclictest1539154-21ssh09:29:002
1490479993729,5cyclictest1513016-21cron08:25:012
1490479993729,5cyclictest1513016-21cron08:25:012
1490479993728,6cyclictest1515530-21/usr/sbin/munin08:34:062
1490479993726,7cyclictest421-21avahi-daemon10:21:142
149047999372,26cyclictest1585847-21ssh10:39:052
1490478993730,4cyclictest1543423-21debian-sa109:35:011
1490478993729,5cyclictest1603042-21cron11:05:011
1490479993629,4cyclictest0-21swapper/209:50:012
1490479993628,5cyclictest422-21dbus-daemon10:15:012
1490479993627,6cyclictest1549104-21ntpq09:44:062
1490479993614,13cyclictest1608869-21ssh11:14:032
149047999361,28cyclictest1545759-21ssh09:39:052
1490479993611,8cyclictest1629808-21sh11:47:252
1490478993630,3cyclictest0-21swapper/112:13:581
1490478993628,5cyclictest0-21swapper/109:19:091
1490478993624,9cyclictest1635366-21/usr/sbin/munin11:55:021
1490477993629,4cyclictest1640838-21missed_timers12:03:560
1490477993629,4cyclictest1632064-21cut11:50:010
1490477993629,4cyclictest1599900-21idleruntime-cro11:00:010
1490477993628,5cyclictest1-21systemd10:05:020
1490477993625,8cyclictest1556120-21switchtime09:54:100
149047999359,19cyclictest1660730-21ssh12:34:012
1490479993526,6cyclictest1605266-21latency11:09:002
1490479993526,5cyclictest1543409-21cron09:35:012
1490479993525,6cyclictest1-21systemd12:28:492
1490479993523,7cyclictest1525903-21ntpq09:08:552
1490478993528,4cyclictest1529987-21munin-plugin-st09:15:011
1490478993527,4cyclictest0-21swapper/110:49:021
1490478993527,3cyclictest0-21swapper/108:59:011
1490478993524,8cyclictest1650864-21timerandwakeup12:19:111
1490477993528,4cyclictest1593357-21munin-plugin-st10:50:010
1490477993526,6cyclictest677358-21cups-browsed12:30:560
1490477993526,5cyclictest12550irq/33-eth009:24:000
1490477993524,8cyclictest0-21swapper/009:45:010
149047799352,10cyclictest1572575-21ssh10:18:590
149047999346,11cyclictest1624800-21cat11:39:142
149047999346,11cyclictest1624800-21cat11:39:142
1490479993428,3cyclictest0-21swapper/208:30:012
1490479993426,5cyclictest1641992-21cron12:05:002
1490479993426,5cyclictest1580196-21munin-plugin-st10:30:012
1490479993425,5cyclictest0-21swapper/212:24:262
1490479993420,8cyclictest1503375-21ntpq07:53:592
1490479993410,7cyclictest1560820-21sh10:01:382
1490478993429,2cyclictest0-21swapper/110:30:011
1490478993426,5cyclictest422-21dbus-daemon12:28:541
1490478993426,5cyclictest1561751-21aten2.4-expect10:03:491
1490478993426,4cyclictest955-21mta-sts-daemon12:38:321
1490478993424,8cyclictest1548988-21ntp_kernel_pll_09:44:021
1490478993424,7cyclictest421-21avahi-daemon12:03:201
1490477993429,3cyclictest191rcuc/009:49:130
1490477993426,2cyclictest0-21swapper/011:20:010
1490477993425,6cyclictest677358-21cups-browsed09:36:050
1490477993423,8cyclictest0-21swapper/009:35:010
1490479993325,5cyclictest1637139-21aten_r2power_vo11:58:502
1490479993322,8cyclictest1501886-21ntp_kernel_pll_07:49:012
1490479993320,8cyclictest1518441-21ntpq08:44:032
149047999331,25cyclictest1589267-21ssh10:44:032
1490478993328,2cyclictest0-21swapper/109:40:011
1490478993326,3cyclictest0-21swapper/111:29:041
1490478993326,3cyclictest0-21swapper/111:24:061
1490478993325,5cyclictest1520512-21munin-run08:50:011
1490478993324,6cyclictest1630452-21aten_r2power_en11:48:511
1490478993321,9cyclictest1497293-21needreboot07:33:581
1490477993325,5cyclictest677334-21cupsd10:24:340
149047799332,27cyclictest151ktimers/008:59:050
149047799332,23cyclictest1664656-21ssh12:39:030
149047999323,23cyclictest1529350-21ssh09:14:072
1490479993226,3cyclictest0-21swapper/211:30:012
1490479993225,3cyclictest0-21swapper/212:13:532
1490479993222,5cyclictest1521505-21ntpq08:54:052
1490479993221,7cyclictest1496135-21swap07:29:082
1490479993221,5cyclictest1592459-21ntpq10:49:032
1490479993219,8cyclictest1516915-21ntpq08:39:002
1490479993218,11cyclictest1650465-21ntpq12:19:022
1490479993217,10cyclictest1581174-21ssh10:32:022
1490479993216,11cyclictest1564352-21ssh10:06:492
1490478993226,3cyclictest0-21swapper/111:19:061
1490478993224,6cyclictest1625416-21sendmail11:40:011
1490478993224,6cyclictest1625416-21sendmail11:40:011
1490478993223,6cyclictest1499459-21run-parts07:40:011
1490477993226,4cyclictest1650690-21systemctl12:19:090
1490477993225,4cyclictest955-21mta-sts-daemon08:29:020
1490477993225,4cyclictest1645050-21munin-plugin-st12:10:010
1490477993225,4cyclictest1568912-21kernelversion10:13:550
1490477993225,4cyclictest1566504-21unixbench_singl10:09:150
1490477993222,8cyclictest0-21swapper/011:54:060
1490477993221,8cyclictest1690079-21ntpd11:24:020
1490479993122,5cyclictest0-21swapper/211:24:012
1490479993119,8cyclictest1494424-21ntpq07:24:062
1490479993119,7cyclictest1491400-21ntpq07:14:012
1490479993116,11cyclictest1620400-21ssh11:33:342
1490479993116,11cyclictest1522946-21ntpq08:59:032
149047899314,12cyclictest1526508-21diskmemload11:56:561
1490478993124,5cyclictest1521994-21cron08:55:011
1490478993124,4cyclictest1620669-21head11:33:481
1490478993124,3cyclictest0-21swapper/108:24:021
1490478993124,3cyclictest0-21swapper/108:24:021
1490478993123,5cyclictest422-21dbus-daemon08:30:011
1490477993125,4cyclictest677358-21cups-browsed12:23:090
1490477993120,9cyclictest0-21swapper/011:13:480
1490477993113,12cyclictest12450irq/32-eth012:28:580
1490479993019,7cyclictest1504870-21ntpq07:59:002
1490479993018,7cyclictest1519923-21ntpq08:48:582
1490479993014,12cyclictest422-21dbus-daemon08:03:592
149047899303,22cyclictest1595928-21ssh10:54:071
1490478993025,2cyclictest0-21swapper/108:40:011
1490478993024,3cyclictest0-21swapper/110:44:021
1490478993023,4cyclictest1500465-21kworker/u8:7-xprtiod08:20:011
1490478993023,4cyclictest1495086-21latency_hist07:25:021
1490478993019,8cyclictest1608523-21kernelversion11:13:551
149047799305,12cyclictest437-21mta-sts-daemon07:50:460
1490477993023,4cyclictest1624004-21fschecks_count11:38:550
1490477993023,4cyclictest1624004-21fschecks_count11:38:550
1490477993022,4cyclictest0-21swapper/007:39:050
149047799302,15cyclictest1618124-21ntpq11:29:020
1490477993020,8cyclictest453-21systemd-logind08:55:020
1490477993017,10cyclictest0-21swapper/009:14:050
1490477993015,11cyclictest151ktimers/008:48:590
1490478992923,3cyclictest0-21swapper/111:08:161
1490478992921,5cyclictest677358-21cups-browsed08:44:241
1490478992921,5cyclictest1503602-21sendmail_mailst07:54:071
1490478992920,7cyclictest698-21cron08:10:011
1490478992920,6cyclictest677358-21cups-browsed07:43:511
1490477992924,3cyclictest1497008-21fschecks_time07:33:540
1490477992923,4cyclictest1500968-21idleruntime-cro07:45:010
1490477992917,9cyclictest0-21swapper/008:05:010
1490477992912,13cyclictest422-21dbus-daemon12:10:310
1490479992816,9cyclictest1509031-21entropy08:13:522
1490479992816,9cyclictest1509031-21entropy08:13:522
1490479992815,9cyclictest1510934-21ntpq08:19:022
149047899284,16cyclictest698-21cron09:29:011
1490478992821,4cyclictest1491961-21idleruntime-cro07:15:011
1490478992820,5cyclictest1507025-21munin-plugin-st08:05:011
1490477992823,3cyclictest1518055-21fschecks_count08:43:550
1490477992822,3cyclictest0-21swapper/009:18:570
1490477992820,5cyclictest1538518-21ssh09:28:490
1490477992819,7cyclictest266-21systemd-journal11:34:060
1490477992819,7cyclictest1606390-21munin-plugin-st11:10:010
1490478992721,3cyclictest0-21swapper/109:50:011
1490477992722,2cyclictest0-21swapper/011:45:020
1490477992719,3cyclictest0-21swapper/010:41:280
1490477992713,3cyclictest0-21swapper/007:13:540
1490478992619,4cyclictest955-21mta-sts-daemon07:29:041
1490477992620,3cyclictest1505673-21latency_hist08:00:020
1490477992620,3cyclictest0-21swapper/011:59:030
149047799261,9cyclictest1582479-21/usr/sbin/munin10:33:560
1490477992616,8cyclictest266-21systemd-journal07:50:010
1490477992615,9cyclictest1-21systemd07:20:010
1490479992516,6cyclictest453-21systemd-logind07:10:012
1490478992520,2cyclictest0-21swapper/108:15:011
1490478992520,2cyclictest0-21swapper/108:15:011
1490478992516,7cyclictest1501984-21sshd07:49:051
1490478992510,12cyclictest271ktimers/108:34:011
149047799251,19cyclictest591-21sshd07:24:060
1490477992419,2cyclictest0-21swapper/008:40:010
1490477992418,4cyclictest1525038-21systemd-run09:05:010
1490477992418,4cyclictest1525038-21systemd-run09:05:010
149047899232,17cyclictest599-21lldpd09:08:541
1490477992318,3cyclictest677358-21cups-browsed08:21:370
1490477992318,3cyclictest677358-21cups-browsed08:21:370
1490477992317,4cyclictest1515998-21cron08:35:010
1490477992218,2cyclictest0-21swapper/009:10:010
1490477992217,3cyclictest989951-21kworker/u8:3-xprtiod07:29:000
1490477992217,2cyclictest0-21swapper/008:18:580
1490478992116,2cyclictest0-21swapper/107:10:021
149048099193,5cyclictest0-21swapper/312:39:013
1490477991910,2cyclictest0-21swapper/007:10:020
149048099183,6cyclictest0-21swapper/310:49:013
149048099173,7cyclictest0-21swapper/309:34:043
149048099163,6cyclictest0-21swapper/310:34:023
149048099162,9cyclictest0-21swapper/309:14:033
149048099162,8cyclictest0-21swapper/311:04:043
149048099162,10cyclictest0-21swapper/309:59:053
149048099161,9cyclictest0-21swapper/310:54:043
149048099153,5cyclictest0-21swapper/307:28:573
149048099152,8cyclictest0-21swapper/310:59:013
149048099152,8cyclictest0-21swapper/309:44:023
149048099152,7cyclictest0-21swapper/312:24:153
149048099152,7cyclictest0-21swapper/310:13:583
149048099151,9cyclictest0-21swapper/310:19:063
149048099151,7cyclictest0-21swapper/311:15:033
149048099151,7cyclictest0-21swapper/307:44:073
149048099151,5cyclictest0-21swapper/307:50:463
149048099143,6cyclictest0-21swapper/311:39:053
149048099143,6cyclictest0-21swapper/311:39:053
149048099142,8cyclictest0-21swapper/311:19:063
149048099142,8cyclictest0-21swapper/308:59:063
149048099142,7cyclictest0-21swapper/312:35:013
149048099142,7cyclictest0-21swapper/311:54:063
149048099142,7cyclictest0-21swapper/311:43:573
149048099142,7cyclictest0-21swapper/310:44:143
149048099142,7cyclictest0-21swapper/310:37:063
149048099142,7cyclictest0-21swapper/309:53:593
149048099142,7cyclictest0-21swapper/309:23:553
149048099142,6cyclictest0-21swapper/308:24:023
149048099142,6cyclictest0-21swapper/308:24:023
149048099142,6cyclictest0-21swapper/308:04:033
149048099141,8cyclictest0-21swapper/312:09:003
149048099141,8cyclictest0-21swapper/311:24:023
149048099141,8cyclictest0-21swapper/310:04:053
149048099141,7cyclictest0-21swapper/312:04:073
149048099141,7cyclictest0-21swapper/309:39:093
149048099141,6cyclictest0-21swapper/308:14:023
149048099141,6cyclictest0-21swapper/308:14:023
149048099133,5cyclictest0-21swapper/309:49:063
149048099132,7cyclictest0-21swapper/312:29:003
149048099132,7cyclictest0-21swapper/311:49:043
149048099131,8cyclictest0-21swapper/308:54:053
149048099131,7cyclictest0-21swapper/311:29:033
149048099131,7cyclictest0-21swapper/311:08:163
149048099131,7cyclictest0-21swapper/310:23:483
149048099131,6cyclictest0-21swapper/309:08:593
149048099131,6cyclictest0-21swapper/308:48:573
149048099131,6cyclictest0-21swapper/308:08:593
149048099131,6cyclictest0-21swapper/307:34:003
149048099131,6cyclictest0-21swapper/307:19:023
149048099123,6cyclictest0-21swapper/310:09:063
149048099122,6cyclictest0-21swapper/312:18:493
149048099122,6cyclictest0-21swapper/311:33:463
149048099122,6cyclictest0-21swapper/309:16:113
149048099122,6cyclictest0-21swapper/307:58:553
149048099122,5cyclictest0-21swapper/312:11:153
149048099122,5cyclictest0-21swapper/308:28:553
149048099122,4cyclictest0-21swapper/308:44:003
149048099121,6cyclictest0-21swapper/309:25:053
149048099121,6cyclictest0-21swapper/309:03:593
149048099121,6cyclictest0-21swapper/309:03:593
149048099121,6cyclictest0-21swapper/307:48:493
149048099112,6cyclictest0-21swapper/310:27:063
149048099112,5cyclictest0-21swapper/308:38:563
149048099112,5cyclictest0-21swapper/307:24:063
149048099101,5cyclictest0-21swapper/311:58:483
149048099101,5cyclictest0-21swapper/308:20:023
149048099101,5cyclictest0-21swapper/307:38:573
14904809991,5cyclictest0-21swapper/308:33:503
14904809991,5cyclictest0-21swapper/307:12:063
14904809991,4cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional