You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-23 - 01:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Wed Apr 22, 2026 12:43:18)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1450698995848,6cyclictest1576253-21sendmail11:00:011
1450699995142,6cyclictest421-21dbus-daemon08:20:012
1450698995042,4cyclictest0-21swapper/111:25:421
1450699994536,5cyclictest1511180-21sendmail09:40:012
1450698994537,5cyclictest421-21dbus-daemon11:25:011
1450697994437,4cyclictest1450696-21cyclictest10:20:390
1450699994336,4cyclictest1549760-21unixbench_multi10:25:532
1450699994336,4cyclictest1549760-21unixbench_multi10:25:532
1450699994335,5cyclictest1657161-21munin-run12:40:002
1450698994335,5cyclictest428-21mta-sts-daemon12:25:011
1450697994335,5cyclictest1527535-21munin-run10:00:000
1450698994234,5cyclictest1657218-21ssh12:40:011
1450698994231,7cyclictest1-21systemd11:35:021
1450699994135,4cyclictest1055-21mta-sts-daemon09:20:382
1450699994134,4cyclictest1527531-21munin-plugin-st10:00:002
1450699994133,5cyclictest1608682-21tr11:40:012
1450697994131,7cyclictest420-21avahi-daemon12:00:430
1450699994033,4cyclictest1483724-21basename09:00:012
1450699994032,5cyclictest1592435-21cron11:20:002
1450698994032,5cyclictest1590248-21unixbench_singl11:15:551
145069899402,28cyclictest1508259-21ssh09:35:401
1450699993931,5cyclictest1650965-21unixbench_singl12:30:532
1450699993931,5cyclictest1650965-21unixbench_singl12:30:532
1450699993931,4cyclictest1492637-21tail09:15:532
1450699993931,4cyclictest1486729-21cut09:10:002
1450698993932,5cyclictest1610522-21unixbench_singl11:40:531
1450698993931,5cyclictest1584401-21ssh11:10:021
1450698993930,5cyclictest1529390-21unixbench_multi10:00:531
1450699993832,3cyclictest0-21swapper/210:05:482
1450698993833,3cyclictest0-21swapper/112:20:011
1450698993829,6cyclictest650582-21cups-browsed10:21:491
1450698993829,6cyclictest1561198-21missed_timers10:40:371
1450698993823,6cyclictest650582-21cups-browsed10:48:581
1450699993731,3cyclictest0-21swapper/210:14:292
1450699993729,5cyclictest1543893-21sendmail10:20:012
1450699993727,6cyclictest420-21avahi-daemon11:50:582
1450698993729,5cyclictest1624796-21latency_hist12:00:011
1450698993727,7cyclictest1485248-21true09:05:011
145069899371,30cyclictest1634125-21ssh12:10:421
1450697993730,5cyclictest420-21avahi-daemon10:15:010
1450697993730,5cyclictest1613495-21fschecks_count11:45:320
1450697993730,4cyclictest1640967-21munin-plugin-st12:20:010
1450697993729,5cyclictest420-21avahi-daemon11:27:080
1450699993630,3cyclictest0-21swapper/211:40:392
1450699993630,3cyclictest0-21swapper/209:25:412
1450699993626,7cyclictest420-21avahi-daemon09:13:022
1450699993626,7cyclictest261-21systemd-journal08:05:022
1450698993629,4cyclictest1549732-21grep10:25:531
1450698993629,4cyclictest1549732-21grep10:25:531
1450698993628,5cyclictest1507031-21cron09:35:011
1450698993625,8cyclictest1500690-21timerwakeupswit09:25:521
1450698993625,7cyclictest420-21avahi-daemon09:13:131
1450698993622,11cyclictest1569866-21timerandwakeup10:50:521
1450697993629,4cyclictest1610519-21ssh11:40:530
1450697993629,4cyclictest1584493-21latency_hist11:10:020
1450697993629,4cyclictest1576269-21awk11:00:010
1450697993626,6cyclictest261-21systemd-journal12:22:330
1450697993619,11cyclictest12450irq/33-eth009:10:400
1450699993528,4cyclictest1572237-21ssh10:55:012
1450699993525,7cyclictest1560112-21sendmail10:40:002
1450698993527,5cyclictest1515267-21idleruntime-cro09:45:011
1450698993522,10cyclictest1613788-21missed_timers11:45:351
1450697993527,5cyclictest420-21avahi-daemon12:25:250
1450697993527,5cyclictest420-21avahi-daemon12:25:250
1450699993428,3cyclictest0-21swapper/211:50:022
1450699993428,3cyclictest0-21swapper/210:20:432
1450699993428,3cyclictest0-21swapper/208:45:542
1450699993426,5cyclictest1560667-21aten_r2power_en10:40:302
1450699993426,5cyclictest1517110-21sed09:45:512
1450699993426,5cyclictest1517110-21sed09:45:512
1450699993424,7cyclictest421-21dbus-daemon12:17:022
1450699993424,7cyclictest420-21avahi-daemon12:01:382
1450698993428,3cyclictest0-21swapper/110:00:011
1450698993427,4cyclictest0-21swapper/109:45:461
1450698993427,4cyclictest0-21swapper/109:45:461
1450698993426,5cyclictest1556036-21cron10:35:001
1450698993426,5cyclictest1556036-21cron10:35:001
1450698993414,15cyclictest650582-21cups-browsed08:20:471
1450697993424,4cyclictest1-21systemd09:00:010
1450697993423,9cyclictest1452634-21cpuspeed07:15:300
1450697993421,11cyclictest1495759-21if_eth009:20:340
1450699993328,2cyclictest0-21swapper/211:25:322
1450699993327,3cyclictest0-21swapper/212:25:012
1450699993327,3cyclictest0-21swapper/210:00:322
1450699993327,3cyclictest0-21swapper/208:40:012
1450699993326,3cyclictest0-21swapper/211:00:402
1450699993325,5cyclictest1456752-21systemd-run07:30:012
1450699993322,8cyclictest1568195-21munin-plugin-st10:50:012
1450699993322,8cyclictest1457631-21ntp_kernel_pll_07:30:422
1450699993322,7cyclictest1624757-21awk12:00:012
1450698993329,2cyclictest0-21swapper/109:20:011
1450698993327,3cyclictest0-21swapper/108:40:011
1450698993326,4cyclictest1479253-21kworker/u8:3-xprtiod09:00:001
1450698993325,5cyclictest1629748-21kernelversion12:05:351
1450698993325,5cyclictest1580325-21grep11:05:011
1450698993325,5cyclictest1055-21mta-sts-daemon12:32:301
1450698993325,5cyclictest1055-21mta-sts-daemon12:32:301
1450698993322,8cyclictest1463536-21needreboot07:50:351
1450697993326,5cyclictest1543873-21munin-run10:20:010
1450697993326,4cyclictest1450708-21systemd-run07:10:020
1450697993326,4cyclictest1450708-21systemd-run07:10:020
1450697993324,7cyclictest1553787-21sed10:30:530
1450697993324,7cyclictest1553787-21sed10:30:530
1450699993227,2cyclictest0-21swapper/209:35:012
1450699993225,3cyclictest0-21swapper/208:50:442
1450699993224,5cyclictest439-21systemd-logind11:15:022
1450699993224,5cyclictest439-21systemd-logind11:15:022
1450699993224,4cyclictest1465718-21munin-plugin-st08:00:002
1450699993223,7cyclictest1054-21mta-sts-daemon10:55:182
145069899325,23cyclictest1605810-21ssh11:35:401
1450698993225,4cyclictest1055-21mta-sts-daemon09:53:351
1450697993225,4cyclictest1653130-21cron12:35:000
1450697993225,4cyclictest1653130-21cron12:35:000
1450697993225,4cyclictest1532160-21aten_r2power_cu10:05:290
1450697993225,4cyclictest1479156-21head08:40:520
1450697993225,3cyclictest0-21swapper/009:25:470
1450697993224,6cyclictest1462734-21cut07:50:020
1450697993224,5cyclictest650579-21cupsd10:38:310
1450697993224,5cyclictest1622173-21mailstats11:55:490
1450697993223,6cyclictest420-21avahi-daemon10:46:060
1450699993124,3cyclictest0-21swapper/212:25:422
1450699993124,3cyclictest0-21swapper/212:25:422
1450699993123,5cyclictest1580817-21telnet11:05:292
1450699993121,7cyclictest1601717-21ntp_kernel_err11:30:402
1450699993120,8cyclictest421-21dbus-daemon07:35:412
1450699993120,8cyclictest1469596-21ntp_kernel_pll_08:10:392
1450698993125,3cyclictest0-21swapper/112:30:011
1450698993125,3cyclictest0-21swapper/112:30:011
1450698993123,5cyclictest421-21dbus-daemon11:15:021
1450698993123,5cyclictest421-21dbus-daemon11:15:021
1450698993123,5cyclictest1453682-21munin-plugin-st07:20:001
1450698993123,5cyclictest1450702-21systemd-run07:10:021
1450698993123,5cyclictest1450702-21systemd-run07:10:021
1450698993122,5cyclictest1482232-21systemd-run08:55:011
1450697993125,4cyclictest1580278-21sh11:04:570
1450697993124,5cyclictest1654566-21mailstats12:35:460
1450697993124,4cyclictest1486726-21diskmemload09:35:460
1450697993124,4cyclictest1457260-21fschecks_time07:30:340
1450697993123,5cyclictest1470236-21systemd-run08:15:010
1450697993123,5cyclictest1054-21mta-sts-daemon08:50:340
1450697993122,6cyclictest421-21dbus-daemon10:00:550
1450697993122,6cyclictest1634518-21timerandwakeup12:10:510
145069999304,7cyclictest1486713-21kworker/u8:4+events_unbound09:40:452
1450699993025,2cyclictest0-21swapper/207:20:002
1450699993024,3cyclictest0-21swapper/212:05:382
1450699993024,3cyclictest0-21swapper/209:50:302
1450699993022,5cyclictest1593261-21fschecks_count11:20:332
145069899302,4cyclictest171rcu_preempt10:10:451
1450698993023,4cyclictest1056-21mta-sts-daemon08:11:481
1450697993024,4cyclictest1512960-21unixbench_multi09:40:520
1450697993024,4cyclictest1486726-21diskmemload11:20:010
1450697993024,4cyclictest1474633-21unixbench_multi08:25:510
1450697993023,5cyclictest420-21avahi-daemon07:25:470
1450697993023,4cyclictest650582-21cups-browsed11:39:580
1450697993023,4cyclictest1467121-21unixbench_multi08:00:510
1450697993022,5cyclictest1054-21mta-sts-daemon07:55:420
1450697993022,2cyclictest0-21swapper/010:45:000
1450697993021,6cyclictest650582-21cups-browsed09:32:390
1450699992922,4cyclictest1051-21mta-sts-daemon10:33:302
1450699992922,4cyclictest1051-21mta-sts-daemon10:33:302
1450699992922,3cyclictest0-21swapper/212:10:522
1450698992923,3cyclictest0-21swapper/110:20:011
1450698992921,5cyclictest1532784-21needreboot10:05:371
145069899291,22cyclictest1617966-21ssh11:50:401
1450697992923,4cyclictest1459767-21systemd-run07:40:010
1450697992923,3cyclictest1491573-21kworker/u8:6-rpciod09:17:260
1450697992923,3cyclictest0-21swapper/010:50:340
1450697992922,3cyclictest0-21swapper/011:50:510
1450697992921,5cyclictest650582-21cups-browsed10:27:420
1450697992921,5cyclictest650582-21cups-browsed10:27:420
1450697992921,5cyclictest12350irq/32-eth009:50:430
1450697992914,3cyclictest171rcu_preempt09:45:500
1450697992914,3cyclictest171rcu_preempt09:45:500
145069999283,6cyclictest171rcu_preempt07:50:382
1450699992821,3cyclictest0-21swapper/209:05:012
1450699992819,7cyclictest1451238-21idleruntime07:10:342
1450698992822,3cyclictest0-21swapper/110:40:001
1450698992820,5cyclictest1468254-21proc_pri08:05:461
1450698992820,5cyclictest1468254-21proc_pri08:05:461
1450698992819,6cyclictest1476212-21cron08:35:011
1450698992818,7cyclictest0-21swapper/112:00:471
1450697992822,4cyclictest421-21dbus-daemon12:05:070
1450697992822,4cyclictest1473149-21unixbench_singl08:20:520
1450697992820,5cyclictest1601013-21/usr/sbin/munin11:30:470
1450697992819,6cyclictest1585261-21if_wlan011:10:380
1450697992819,6cyclictest1585261-21if_wlan011:10:380
1450698992721,3cyclictest0-21swapper/107:35:421
1450698992719,5cyclictest1471662-21grep08:15:511
1450697992722,3cyclictest0-21swapper/011:20:430
1450697992721,3cyclictest1477201-21ntpq08:35:450
1450697992721,3cyclictest0-21swapper/007:50:320
1450697992720,4cyclictest1486722-21idleruntime-cro09:10:000
1450697992718,6cyclictest0-21swapper/007:45:010
1450699992620,3cyclictest0-21swapper/208:30:012
1450698992621,2cyclictest0-21swapper/108:00:011
1450698992620,3cyclictest0-21swapper/109:05:371
1450698992620,3cyclictest0-21swapper/108:45:331
1450698992619,3cyclictest0-21swapper/109:20:411
1450698992618,5cyclictest1479221-21cron08:45:011
1450698992614,8cyclictest261-21systemd-journal07:30:321
1450697992619,5cyclictest650582-21cups-browsed09:00:450
1450697992619,4cyclictest261-21systemd-journal08:35:010
1450697992616,7cyclictest1471394-21switchtime08:15:490
1450699992517,5cyclictest439-21systemd-logind08:25:012
1450699992517,5cyclictest1479251-21systemd-run08:45:012
1450698992517,5cyclictest1461132-21tail07:40:501
145069999244,14cyclictest428-21mta-sts-daemon07:40:442
1450699992416,5cyclictest1455188-21cron07:25:012
1450698992418,3cyclictest1054-21mta-sts-daemon07:27:041
1450698992416,5cyclictest1452200-21systemd-run07:15:011
145069899241,17cyclictest650582-21cups-browsed07:45:461
1450697992418,4cyclictest1454719-21proc_pri07:20:470
145070099238,7cyclictest0-21swapper/311:10:463
145070099238,7cyclictest0-21swapper/311:10:463
145069999232,16cyclictest1461460-21/usr/sbin/munin07:45:542
1450699992316,4cyclictest1467566-21aten_r2power_fr08:05:302
1450699992316,4cyclictest1467566-21aten_r2power_fr08:05:302
1450698992317,3cyclictest0-21swapper/108:00:421
1450698992315,5cyclictest1455091-21unixbench_multi07:20:521
145069799232,15cyclictest1451628-21ntpq07:10:430
1450697992314,5cyclictest650582-21cups-browsed08:45:430
145070099228,7cyclictest0-21swapper/311:20:423
145069999222,15cyclictest1475689-21ntpq08:30:452
1450698992216,3cyclictest0-21swapper/108:25:431
1450697992214,5cyclictest538-21wpa_supplicant08:05:510
1450697992214,5cyclictest538-21wpa_supplicant08:05:510
145070099219,7cyclictest0-21swapper/308:00:393
145070099217,8cyclictest0-21swapper/309:40:463
145070099217,7cyclictest0-21swapper/309:35:473
145070099207,7cyclictest0-21swapper/311:05:453
145070099207,6cyclictest0-21swapper/309:20:383
145070099207,6cyclictest0-21swapper/309:15:433
145070099206,8cyclictest0-21swapper/310:35:413
145070099206,6cyclictest0-21swapper/311:30:423
145070099198,5cyclictest0-21swapper/307:35:383
145070099197,6cyclictest0-21swapper/310:30:393
145070099197,6cyclictest0-21swapper/310:30:393
145070099197,6cyclictest0-21swapper/309:10:403
145070099197,5cyclictest0-21swapper/311:35:413
145070099196,7cyclictest0-21swapper/312:00:393
145070099196,7cyclictest0-21swapper/310:50:403
145070099193,10cyclictest0-21swapper/311:00:423
145070099187,7cyclictest0-21swapper/309:55:433
145070099187,6cyclictest0-21swapper/310:15:403
145070099187,6cyclictest0-21swapper/310:10:423
145070099187,6cyclictest0-21swapper/310:05:453
145070099182,10cyclictest0-21swapper/310:45:403
145070099181,11cyclictest0-21swapper/309:25:413
145070099178,5cyclictest0-21swapper/312:30:393
145070099178,5cyclictest0-21swapper/312:30:393
145070099178,5cyclictest0-21swapper/309:45:393
145070099178,5cyclictest0-21swapper/309:45:393
145070099178,5cyclictest0-21swapper/307:10:393
145070099177,6cyclictest0-21swapper/311:25:423
145070099177,5cyclictest0-21swapper/312:25:483
145070099177,5cyclictest0-21swapper/312:25:483
145070099177,5cyclictest0-21swapper/307:40:393
145070099177,5cyclictest0-21swapper/307:15:383
145070099176,7cyclictest0-21swapper/312:35:333
145070099176,6cyclictest0-21swapper/310:40:433
145070099176,6cyclictest0-21swapper/309:30:453
145070099175,7cyclictest0-21swapper/310:20:393
145070099173,8cyclictest0-21swapper/309:10:003
145070099172,9cyclictest0-21swapper/310:25:433
145070099172,9cyclictest0-21swapper/310:25:433
145070099171,10cyclictest0-21swapper/312:05:373
145070099168,4cyclictest0-21swapper/309:00:393
145070099168,4cyclictest0-21swapper/308:50:453
145070099168,4cyclictest0-21swapper/308:40:423
145070099168,4cyclictest0-21swapper/307:55:413
145070099168,4cyclictest0-21swapper/307:28:003
145070099167,6cyclictest0-21swapper/308:10:423
145070099167,5cyclictest0-21swapper/312:10:223
145070099167,5cyclictest0-21swapper/308:35:383
145070099167,5cyclictest0-21swapper/308:20:453
145070099167,4cyclictest0-21swapper/309:50:433
145070099167,4cyclictest0-21swapper/307:50:363
145070099166,6cyclictest0-21swapper/311:15:323
145070099166,6cyclictest0-21swapper/310:55:453
145070099163,8cyclictest0-21swapper/312:20:373
145070099162,9cyclictest0-21swapper/311:45:453
145070099162,8cyclictest0-21swapper/312:15:413
145070099161,9cyclictest0-21swapper/311:55:313
1450700991612,2cyclictest0-21swapper/307:10:013
1450700991612,2cyclictest0-21swapper/307:10:013
145070099161,10cyclictest0-21swapper/310:00:373
1450700991610,4cyclictest0-21swapper/311:40:333
145069999169,4cyclictest1450875-21latency_hist07:10:032
145069999169,4cyclictest1450875-21latency_hist07:10:032
145070099159,3cyclictest0-21swapper/308:30:403
145070099158,4cyclictest0-21swapper/308:55:453
145070099158,4cyclictest0-21swapper/308:25:403
145070099158,4cyclictest0-21swapper/308:15:383
145070099157,4cyclictest0-21swapper/307:45:403
145070099151,9cyclictest0-21swapper/311:50:253
145070099149,3cyclictest0-21swapper/308:05:423
145070099149,3cyclictest0-21swapper/308:05:423
145070099148,4cyclictest0-21swapper/308:45:413
145070099148,4cyclictest0-21swapper/307:20:453
145070099147,4cyclictest0-21swapper/307:30:393
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional