You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-22 - 04:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Wed Apr 22, 2026 00:43:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
478961998669,11cyclictest659789-21ssh00:10:450
478963998475,3cyclictest0-21swapper/200:11:032
478962996550,12cyclictest659806-21ssh00:10:461
478963996147,9cyclictest420-21avahi-daemon22:50:402
478963995742,10cyclictest2852470-21cups-browsed21:30:402
478963995742,10cyclictest2852470-21cups-browsed21:30:402
478963995138,9cyclictest587026-21sendmail22:40:012
478961995120,26cyclictest0-21swapper/023:45:410
478962995025,22cyclictest501467-21cron20:25:001
478963994938,7cyclictest619603-21df_inode23:20:342
478961994938,7cyclictest151ktimers/021:25:440
478961994917,29cyclictest682258-21idleruntime-cro00:40:000
478963994836,9cyclictest0-21swapper/222:40:342
478962994838,7cyclictest571999-21/usr/sbin/munin22:20:351
478961994835,10cyclictest0-21swapper/021:20:010
47896299477,33cyclictest0-21swapper/121:43:351
478961994740,4cyclictest650584-21cron00:00:000
478961994738,6cyclictest551248-21systemd21:55:020
47896399463,34cyclictest592237-21ssh22:45:412
47896299464,34cyclictest627965-21ssh23:30:391
47896299462,38cyclictest560507-21ssh22:05:471
47896299462,16cyclictest651872-21ntpq00:00:431
478961994614,29cyclictest524877-21unixbench_singl21:20:540
478963994535,6cyclictest531021-21cron21:30:012
478963994534,7cyclictest2852470-21cups-browsed23:52:202
478963994534,7cyclictest2852470-21cups-browsed23:52:202
478963994433,7cyclictest420-21avahi-daemon22:59:172
47896299442,7cyclictest488890-21ntpq19:40:421
478961994424,17cyclictest479099-21latency_hist19:10:020
478963994336,4cyclictest571161-21sendmail22:20:012
478963994336,4cyclictest571161-21sendmail22:20:012
478962994333,7cyclictest623632-21gltestperf23:25:341
47896299432,6cyclictest484366-21ntpq19:25:401
47896299432,14cyclictest532198-21ntpq21:30:391
47896299432,14cyclictest532198-21ntpq21:30:391
478962994319,18cyclictest0-21swapper/123:55:021
478962994319,18cyclictest0-21swapper/123:55:021
478961994335,5cyclictest1-21systemd22:35:010
478961994314,9cyclictest421-21dbus-daemon23:30:290
478963994233,4cyclictest0-21swapper/223:35:482
478962994234,5cyclictest650556-40logrotate00:00:001
478962994234,5cyclictest614874-21cron23:15:001
47896299422,6cyclictest509901-21ntpq20:50:431
47896299422,6cyclictest493352-21ntpq19:55:441
47896299422,14cyclictest655856-21ntpq00:05:441
478962994221,18cyclictest0-21swapper/100:30:351
47896299421,9cyclictest506927-21ntpq20:40:431
47896299421,4cyclictest616057-21ntpq23:15:431
47896299421,11cyclictest664787-21ssh00:16:501
478961994230,5cyclictest151ktimers/019:20:390
478961994230,5cyclictest151ktimers/019:20:390
478961994220,19cyclictest532228-21sh21:30:400
478961994220,19cyclictest532228-21sh21:30:400
478961994219,19cyclictest0-21swapper/022:15:370
478961994219,19cyclictest0-21swapper/022:15:370
47896199421,22cyclictest600156-21ntpq22:55:410
47896199421,15cyclictest499480-21ps20:15:460
47896199421,15cyclictest499480-21ps20:15:460
47896199421,11cyclictest675269-21ssh00:30:350
47896399412,31cyclictest604285-21ssh23:00:442
47896299413,32cyclictest620107-21ntpq23:20:431
47896299413,30cyclictest632092-21ssh23:35:471
47896299412,4cyclictest497848-21ntpq20:10:401
47896299412,4cyclictest497848-21ntpq20:10:401
47896299412,3cyclictest600173-21ntpq22:55:411
47896299411,14cyclictest552503-21ssh21:55:441
47896299411,11cyclictest505414-21ntpq20:35:451
47896299411,10cyclictest596290-21ntpq22:50:421
47896199415,6cyclictest516157-21ssh21:10:420
478961994135,4cyclictest624569-21unixbench-2d23:25:530
478961994132,6cyclictest2852470-21cups-browsed23:35:590
478961994130,8cyclictest595617-21aten_r2power_vo22:50:290
47896199412,15cyclictest506196-21/usr/sbin/munin20:40:530
47896199412,14cyclictest667738-21ntp_states00:20:460
47896199412,14cyclictest667738-21ntp_states00:20:460
47896199412,13cyclictest643971-21ntpq23:50:450
47896199412,13cyclictest643971-21ntpq23:50:450
478961994118,20cyclictest650582-21cups-browsed00:05:370
478961994113,25cyclictest439-21systemd-logind19:15:010
478963994032,4cyclictest674367-21systemd-run00:30:012
478963994032,4cyclictest674367-21systemd-run00:30:012
478963994031,6cyclictest420-21avahi-daemon20:25:472
478963994031,4cyclictest0-21swapper/223:15:462
47896299404,30cyclictest508347-21ntpq20:45:391
478962994033,4cyclictest440446-21kworker/u8:8-xprtiod19:10:011
478962994030,5cyclictest0-21swapper/122:25:421
47896299402,8cyclictest491974-21ps19:50:461
47896299402,5cyclictest667657-21ntpq00:20:421
47896299402,5cyclictest667657-21ntpq00:20:421
47896299402,5cyclictest511330-21ntpq20:55:401
47896299402,5cyclictest496312-21ntpq20:05:391
47896299402,4cyclictest494931-21ntpq20:00:451
47896299402,3cyclictest502402-21ntpq20:25:431
47896299401,9cyclictest609280-21ssh23:06:441
47896299401,9cyclictest517684-21ssh21:12:271
47896299401,16cyclictest588452-21ps22:40:481
47896299401,11cyclictest638099-21ssh23:43:541
47896299401,10cyclictest543398-21ssh21:45:041
47896299401,10cyclictest504107-21/usr/sbin/munin20:30:491
47896299401,10cyclictest499421-21ntpq20:15:441
47896299401,10cyclictest499421-21ntpq20:15:441
47896199402,5cyclictest588274-21awk22:40:410
478961994019,18cyclictest615482-21aten2.4-expect23:15:330
478961994017,2cyclictest0-21swapper/022:00:440
478961994014,24cyclictest671413-21missed_timers00:25:360
478961994014,24cyclictest671413-21missed_timers00:25:360
478961994013,3cyclictest1056-21mta-sts-daemon23:04:390
478961994013,3cyclictest0-21swapper/000:00:330
47896199401,13cyclictest492951-21diskstats19:55:350
478963993934,2cyclictest0-21swapper/200:00:002
478963993931,5cyclictest611090-21latency_hist23:10:032
478963993931,5cyclictest490976-21systemd-run19:50:012
478963993931,5cyclictest484967-21systemd-run19:30:012
478963993930,6cyclictest575155-21unin-run22:25:012
478963993930,6cyclictest543350-21latency_hist21:45:022
478963993930,6cyclictest0-21swapper/200:40:012
478963993929,6cyclictest670551-21latency_hist00:25:032
478963993929,6cyclictest670551-21latency_hist00:25:032
478963993929,6cyclictest421-21dbus-daemon22:30:402
47896299397,29cyclictest521502-21ssh21:17:051
47896299393,6cyclictest481369-21ntpq19:15:431
47896299392,7cyclictest564420-21ntpq22:10:451
47896299392,6cyclictest479865-21ntpq19:10:411
47896299392,5cyclictest487341-21ntpq19:35:401
47896299391,5cyclictest524375-21ntpq21:20:471
47896299391,3cyclictest482855-21ntpq19:20:391
47896299391,3cyclictest482855-21ntpq19:20:391
47896299391,14cyclictest679468-21ntpq00:35:431
47896299391,10cyclictest640880-21ssh23:46:331
47896299391,10cyclictest606565-21ssh23:04:141
47896299391,10cyclictest585713-21ssh22:37:251
47896299391,10cyclictest557483-21ssh22:01:351
47896199394,10cyclictest497932-21ntpq20:10:450
47896199394,10cyclictest497932-21ntpq20:10:450
478961993933,4cyclictest608093-21needreboot23:05:390
47896199392,15cyclictest659-21snmpd22:10:490
478961993915,4cyclictest514957-21diskmemload22:05:310
478961993914,5cyclictest261-21systemd-journal19:30:000
478963993830,4cyclictest0-21swapper/200:00:542
478963993829,6cyclictest2852470-21cups-browsed22:25:432
478963993826,9cyclictest628484-21timerwakeupswit23:30:512
47896299382,8cyclictest485810-21ntpq19:30:391
47896299382,4cyclictest514371-21ntpq21:05:421
47896299381,9cyclictest671116-21df00:25:311
47896299381,9cyclictest671116-21df00:25:311
47896299381,9cyclictest591395-21ssh22:45:191
47896299381,9cyclictest568498-21ntpq22:15:451
47896299381,9cyclictest568498-21ntpq22:15:451
47896299381,8cyclictest490384-21ntpq19:45:411
47896299381,3cyclictest528275-21ntpq21:25:431
47896299381,10cyclictest582337-21ssh22:33:371
47896299381,10cyclictest547495-21ssh21:50:111
47896299381,10cyclictest535444-21ssh21:35:271
47896199389,6cyclictest636003-21diskmemload23:40:430
47896199387,10cyclictest0-21swapper/023:10:510
478961993814,5cyclictest0-21swapper/021:40:310
478961993813,2cyclictest0-21swapper/000:15:300
478961993812,5cyclictest0-21swapper/020:45:510
47896199381,21cyclictest552444-21ntpq21:55:420
478961993812,19cyclictest584291-21sh22:35:420
47896199381,16cyclictest509874-21ntpq20:50:420
47896199381,15cyclictest495019-21ps20:00:470
47896199381,14cyclictest576401-21ntpq22:25:440
478963993727,7cyclictest663071-21/usr/sbin/munin00:15:312
478963993724,3cyclictest0-21swapper/221:15:382
47896299371,4cyclictest512792-21/usr/sbin/munin21:00:381
47896199373,9cyclictest620194-21ntpq23:20:470
478961993730,4cyclictest513487-21systemd-run21:05:010
478961993714,2cyclictest0-21swapper/021:35:440
47896199371,18cyclictest592182-21ntpq22:45:390
47896199371,16cyclictest572479-21ntpq22:20:450
478961993711,6cyclictest487848-21unixbench_multi19:35:510
478961993710,5cyclictest501407-21unixbench_singl20:20:520
478963993629,4cyclictest1056-21mta-sts-daemon23:15:012
478963993628,4cyclictest0-21swapper/221:55:462
478963993627,6cyclictest420-21avahi-daemon00:30:462
47896399362,29cyclictest560443-21ssh22:05:442
47896399361,28cyclictest544405-21ssh21:45:422
478961993615,5cyclictest420-21avahi-daemon20:25:310
478961993614,2cyclictest0-21swapper/019:15:300
478961993612,3cyclictest1051-21mta-sts-daemon21:05:200
47896199361,14cyclictest488992-21ps19:40:460
47896199361,11cyclictest543912-21df_inode21:45:330
47896199360,8cyclictest486010-21sendmail_mailqu19:30:470
478963993529,3cyclictest0-21swapper/219:40:012
478961993519,13cyclictest0-21swapper/020:55:460
478963993429,2cyclictest0-21swapper/219:25:012
478963993429,2cyclictest0-21swapper/219:25:012
478963993426,5cyclictest1-21systemd23:50:022
478963993325,4cyclictest0-21swapper/221:50:442
478963993325,4cyclictest0-21swapper/219:20:012
478963993325,3cyclictest0-21swapper/221:20:322
47896399331,25cyclictest564509-21ssh22:10:482
478963993224,5cyclictest510781-21aten_r2power_fr20:55:312
478963993222,8cyclictest489274-21timerwakeupswit19:40:502
47896399321,25cyclictest556451-21ssh22:00:432
47896199321,14cyclictest2852470-21cups-browsed20:30:390
47896399316,14cyclictest516175-21ssh21:10:422
478963993125,3cyclictest0-21swapper/221:40:002
478963993124,3cyclictest0-21swapper/221:05:342
478963993123,5cyclictest420-21avahi-daemon23:27:432
478963993122,6cyclictest493984-21systemd-run20:00:012
47896399312,22cyclictest753-21ntpd20:10:402
47896399312,22cyclictest753-21ntpd20:10:402
478963993121,6cyclictest1055-21mta-sts-daemon00:07:052
47896399311,23cyclictest635911-21ssh23:40:402
478963993023,3cyclictest0-21swapper/220:40:012
478963992922,5cyclictest492456-21munin-run19:55:012
478963992922,3cyclictest0-21swapper/220:30:342
478963992819,7cyclictest508481-21proc_pri20:45:452
47896399273,16cyclictest428-21mta-sts-daemon19:30:442
478963992719,5cyclictest0-21swapper/220:19:162
478963992719,5cyclictest0-21swapper/220:19:162
47896399262,17cyclictest441-21gpm21:00:402
478963992618,5cyclictest496712-21timerandwakeup20:05:512
478961992619,5cyclictest495961-21sshd20:05:330
47896399252,18cyclictest421-21dbus-daemon20:20:442
478963992518,4cyclictest495581-21latency_hist20:05:022
478961992519,4cyclictest1056-21mta-sts-daemon19:50:420
478961992514,8cyclictest261-21systemd-journal20:35:330
478963992417,5cyclictest6675-21kworker/u8:10+xprtiod19:15:012
478963992317,4cyclictest509472-21kworker/u8:3+nfsiod20:53:342
478963992313,7cyclictest478916-21/usr/sbin/munin19:10:012
478961992318,3cyclictest0-21swapper/019:50:010
47896499213,12cyclictest0-21swapper/300:10:453
47896399212,14cyclictest506889-21ntpq20:40:412
47896499193,8cyclictest0-21swapper/323:35:483
47896499193,6cyclictest0-21swapper/322:20:433
47896499192,10cyclictest0-21swapper/300:05:403
47896499184,7cyclictest0-21swapper/322:10:483
47896499183,8cyclictest0-21swapper/300:30:403
47896499173,9cyclictest0-21swapper/323:10:423
47896499172,9cyclictest0-21swapper/322:50:413
47896499172,8cyclictest0-21swapper/300:15:433
47896499172,6cyclictest0-21swapper/320:50:433
47896499171,8cyclictest0-21swapper/322:55:403
47896499171,6cyclictest0-21swapper/320:30:413
47896499162,9cyclictest0-21swapper/321:45:403
47896499162,8cyclictest0-21swapper/321:40:463
47896499162,6cyclictest0-21swapper/319:40:423
47896499153,7cyclictest0-21swapper/323:00:433
47896499153,7cyclictest0-21swapper/322:05:533
47896499153,7cyclictest0-21swapper/321:30:403
47896499153,7cyclictest0-21swapper/321:30:403
47896499151,8cyclictest0-21swapper/323:25:393
47896499151,8cyclictest0-21swapper/323:15:473
47896499151,8cyclictest0-21swapper/321:35:403
47896499151,8cyclictest0-21swapper/321:25:433
47896499151,7cyclictest0-21swapper/323:30:413
47896499151,7cyclictest0-21swapper/321:00:453
47896499151,7cyclictest0-21swapper/300:35:423
47896499151,7cyclictest0-21swapper/300:20:393
47896499151,7cyclictest0-21swapper/300:20:393
47896499151,6cyclictest0-21swapper/320:45:403
47896499142,8cyclictest0-21swapper/321:55:403
47896499142,7cyclictest0-21swapper/323:50:453
47896499142,7cyclictest0-21swapper/323:50:453
47896499142,7cyclictest0-21swapper/321:10:013
47896499142,7cyclictest0-21swapper/300:25:333
47896499142,7cyclictest0-21swapper/300:25:333
47896499142,5cyclictest0-21swapper/320:05:433
47896499141,8cyclictest0-21swapper/323:55:383
47896499141,8cyclictest0-21swapper/323:40:413
47896499141,8cyclictest0-21swapper/322:45:403
47896499141,8cyclictest0-21swapper/321:20:433
47896499141,7cyclictest0-21swapper/323:20:413
47896499141,7cyclictest0-21swapper/323:05:413
47896499141,7cyclictest0-21swapper/321:15:433
47896499141,6cyclictest0-21swapper/320:55:413
47896499141,6cyclictest0-21swapper/320:10:393
47896499141,6cyclictest0-21swapper/320:10:393
47896499141,6cyclictest0-21swapper/319:50:423
47896499133,6cyclictest0-21swapper/322:15:453
47896499133,6cyclictest0-21swapper/322:15:453
47896499132,6cyclictest0-21swapper/322:35:403
47896499132,5cyclictest0-21swapper/322:00:393
47896499132,5cyclictest0-21swapper/319:55:483
47896499132,5cyclictest0-21swapper/319:30:393
47896499131,8cyclictest0-21swapper/322:40:473
47896499131,7cyclictest0-21swapper/323:45:393
47896499131,7cyclictest0-21swapper/322:25:403
47896499131,7cyclictest0-21swapper/321:50:473
47896499131,7cyclictest0-21swapper/321:10:413
47896499131,7cyclictest0-21swapper/300:00:403
47896499131,6cyclictest0-21swapper/320:40:403
47896499131,6cyclictest0-21swapper/320:35:443
47896499131,6cyclictest0-21swapper/319:15:423
47896499131,5cyclictest0-21swapper/319:35:383
47896499131,5cyclictest0-21swapper/319:10:403
47896499122,7cyclictest0-21swapper/322:30:383
47896499122,6cyclictest0-21swapper/320:00:343
47896499122,5cyclictest0-21swapper/320:30:023
47896499122,4cyclictest0-21swapper/319:20:393
47896499122,4cyclictest0-21swapper/319:20:393
47896499121,5cyclictest0-21swapper/320:15:393
47896499121,5cyclictest0-21swapper/320:15:393
47896499121,5cyclictest0-21swapper/319:45:303
47896499101,6cyclictest0-21swapper/319:25:303
4789649991,5cyclictest0-21swapper/320:20:343
4789649981,4cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional