You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-18 - 21:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot3s.osadl.org (updated Sun May 18, 2025 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1443902998884,2cyclictest0-21swapper/008:45:020
1443902998724,24cyclictest1453573-21date10:15:010
1443902998618,27cyclictest0-21swapper/010:25:020
1443902997936,38cyclictest1460311-21sshd12:26:370
1443907997831,11cyclictest235-21systemd-journal11:42:361
1443902997323,16cyclictest1451352-21munin-run09:35:000
1443902997113,25cyclictest0-21swapper/007:55:010
1443902996830,34cyclictest0-21swapper/012:39:550
1443902996345,15cyclictest1444084-21sshd07:14:360
1443902996339,17cyclictest1-21systemd11:30:010
1443902996247,10cyclictest1456983-21sshd11:22:360
1443902996153,4cyclictest1448216-21munin-run08:35:000
1443902996147,10cyclictest1451612-21/usr/sbin/munin09:40:010
1443902996147,10cyclictest1451612-21/usr/sbin/munin09:40:010
1443902996145,12cyclictest1455438-21sshd10:54:360
1443902996143,14cyclictest1448436-21sshd08:36:360
1443902996047,9cyclictest1445880-21sshd07:46:360
1443902996042,14cyclictest1458762-21sshd11:59:360
1443902996038,17cyclictest1460066-21modprobe12:22:380
1443902996017,29cyclictest0-21swapper/010:05:020
1443902995952,4cyclictest1455673-21sshd10:56:360
1443902995946,10cyclictest1456713-21sshd11:19:360
1443902995945,11cyclictest1449455-21sshd08:55:370
1443902995945,10cyclictest1457732-21sshd11:37:360
1443902995944,11cyclictest1449788-21sshd09:01:360
1443902995937,17cyclictest1458514-21sshd11:54:360
1443902995846,9cyclictest1447152-21sshd08:11:360
1443902995846,9cyclictest1444316-21sshd07:15:350
1443902995845,9cyclictest363-21snmpd07:31:000
1443902995845,9cyclictest363-21snmpd07:31:000
1443902995845,10cyclictest1444867-21sshd07:29:360
1443902995844,11cyclictest1448957-21sshd08:45:360
1443902995844,10cyclictest1456239-21munin-run11:10:010
1443902995818,7cyclictest0-21swapper/009:50:020
1443902995744,10cyclictest1450822-21sshd09:20:360
1443902995744,10cyclictest1450822-21sshd09:20:360
1443902995743,10cyclictest1459568-21munin-run12:15:010
1443902995647,4cyclictest1456475-21munin-run11:15:000
1443902995646,6cyclictest1449238-21munin-run08:55:010
1443902995645,8cyclictest1447951-21sshd08:29:360
1443902995645,8cyclictest1447951-21sshd08:29:360
1443902995644,9cyclictest1447691-21sshd08:22:360
1443902995643,9cyclictest1459306-21munin-run12:10:000
1443902995643,9cyclictest1450300-21sshd09:13:360
1443902995642,11cyclictest1457484-21sshd11:32:360
1443902995642,11cyclictest1457484-21sshd11:32:360
1443902995642,10cyclictest1454162-21sshd10:28:360
1443902995642,10cyclictest1452376-21munin-run09:55:010
1443902995639,13cyclictest363-21snmpd11:00:370
1443902995639,13cyclictest363-21snmpd11:00:360
1443902995631,19cyclictest1459863-21cat12:20:020
1443902995625,26cyclictest272-21dbus-daemon10:10:000
1443902995547,4cyclictest1450586-21munin-run09:20:000
1443902995547,4cyclictest1445424-21munin-run07:40:010
1443902995543,9cyclictest1451083-21sshd09:28:360
1443902995540,11cyclictest1444636-21munin-run07:25:000
1443902995538,13cyclictest363-21snmpd10:40:120
1443902995522,29cyclictest150-21pr/ttyS207:58:360
1443907995436,13cyclictest1457762-21cat11:40:001
1443902995446,4cyclictest1451850-21sshd09:41:370
1443902995446,4cyclictest1445643-21sshd07:44:370
1443902995445,6cyclictest1455172-21sshd10:45:360
1443902995445,6cyclictest1455172-21sshd10:45:360
1443902995433,17cyclictest1458250-21sshd11:45:360
1443902995433,17cyclictest1458250-21sshd11:45:360
144391699535,5cyclictest131rcu_preempt10:40:003
1443902995346,4cyclictest1447423-21sshd08:19:360
1443902995335,13cyclictest363-21snmpd10:30:210
1443916995221,26cyclictest37-21ksoftirqd/311:45:003
1443902995245,4cyclictest1452591-21sshd09:55:360
1443902995242,7cyclictest1454647-21sshd10:35:360
1443902995241,8cyclictest1460565-21sshd12:32:370
1443902995240,8cyclictest1446917-21sshd08:09:360
1443912995129,20cyclictest291rcuc/208:35:012
1443907995134,12cyclictest235-21systemd-journal10:30:001
1443902995137,10cyclictest363-21snmpd12:04:480
1443902995137,10cyclictest363-21snmpd12:04:480
1443902995134,14cyclictest363-21snmpd09:05:090
1443902995122,23cyclictest1443981-21latency_hist07:10:020
1443916995033,12cyclictest1-21systemd12:05:013
1443916995033,12cyclictest1-21systemd12:05:013
1443912995034,11cyclictest1-21systemd07:50:002
1443912995013,5cyclictest131rcu_preempt10:50:002
1443912995013,5cyclictest131rcu_preempt10:50:002
1443912995013,5cyclictest131rcu_preempt10:45:002
1443907995033,12cyclictest1-21systemd07:55:011
1443907995010,25cyclictest23-21ksoftirqd/112:40:021
1443902995032,13cyclictest1446648-21modprobe08:00:030
1443916994932,12cyclictest37-21ksoftirqd/308:40:003
1443912994830,13cyclictest235-21systemd-journal11:50:002
1443912994830,13cyclictest235-21systemd-journal11:50:002
1443907994835,11cyclictest23-21ksoftirqd/110:40:011
1443907994833,11cyclictest1456734-21cat11:20:001
1443902994837,8cyclictest363-21snmpd11:42:110
1443902994829,15cyclictest1453637-21modprobe10:18:360
1443902994829,15cyclictest1453637-21modprobe10:18:360
1443916994731,12cyclictest235-21systemd-journal11:50:033
1443916994729,13cyclictest235-21systemd-journal11:05:033
1443916994728,15cyclictest1-21systemd09:05:013
1443907994735,7cyclictest1456224-21munin-plugin-st11:10:001
1443907994728,14cyclictest272-21dbus-daemon08:30:011
1443907994728,14cyclictest272-21dbus-daemon08:30:011
1443907994717,16cyclictest1443898-21cyclictest07:25:021
1443916994630,12cyclictest1-21systemd11:50:013
1443916994630,12cyclictest1-21systemd11:50:013
144391699462,6cyclictest131rcu_preempt11:00:003
1443916994626,14cyclictest272-21dbus-daemon08:15:013
1443916994625,15cyclictest272-21dbus-daemon10:55:003
1443912994633,9cyclictest235-21systemd-journal10:00:002
1443907994632,10cyclictest235-21systemd-journal12:33:371
144391699455,7cyclictest131rcu_preempt11:20:003
1443916994531,10cyclictest235-21systemd-journal10:10:003
1443916994422,16cyclictest272-21dbus-daemon10:45:013
144391699441,7cyclictest235-21systemd-journal08:19:593
1443912994427,13cyclictest272-21dbus-daemon11:05:012
1443912994427,13cyclictest272-21dbus-daemon11:05:012
1443907994426,14cyclictest272-21dbus-daemon12:05:011
1443907994426,14cyclictest272-21dbus-daemon12:05:011
1443916994334,5cyclictest1454182-21munin-plugin-st10:30:003
1443916994331,8cyclictest1-21systemd10:35:013
1443916994326,12cyclictest1-21systemd08:25:013
144391699431,6cyclictest131rcu_preempt12:25:013
1443916994315,6cyclictest131rcu_preempt12:15:013
1443912994329,10cyclictest272-21dbus-daemon08:55:012
1443912994326,13cyclictest235-21systemd-journal11:06:362
1443912994320,6cyclictest30-21ksoftirqd/207:24:592
1443907994332,6cyclictest1443909-21cat07:10:011
1443907994325,13cyclictest235-21systemd-journal08:10:001
1443916994233,5cyclictest306-21in:imuxsock10:50:013
1443916994233,5cyclictest306-21in:imuxsock10:50:013
1443916994232,7cyclictest37-21ksoftirqd/307:40:023
144391699422,6cyclictest131rcu_preempt09:30:023
1443912994230,8cyclictest235-21systemd-journal12:36:372
144391299422,8cyclictest1456902-21sort11:20:022
1443912994227,10cyclictest306-21in:imuxsock09:10:012
1443912994223,14cyclictest1-21systemd12:10:012
144391299421,5cyclictest131rcu_preempt10:05:012
1443907994227,11cyclictest0-21swapper/112:20:011
1443907994224,14cyclictest23-21ksoftirqd/109:35:001
144391699415,5cyclictest361rcuc/308:30:013
144391699415,5cyclictest361rcuc/308:30:013
144391699412,5cyclictest131rcu_preempt07:45:003
1443916994121,15cyclictest272-21dbus-daemon09:00:013
1443916994115,19cyclictest131rcu_preempt09:40:013
1443916994115,19cyclictest131rcu_preempt09:40:013
144391699411,4cyclictest131rcu_preempt12:30:013
144391299411,7cyclictest131rcu_preempt07:30:012
1443907994130,5cyclictest306-21in:imuxsock07:30:001
1443907994130,5cyclictest1453932-21munin-plugin-st10:25:001
1443907994127,9cyclictest1-21systemd10:00:001
1443907994126,11cyclictest272-21dbus-daemon12:25:501
144390799411,3cyclictest131rcu_preempt10:15:011
144391699406,5cyclictest131rcu_preempt09:20:013
144391699405,3cyclictest0-21swapper/311:35:023
144391699405,3cyclictest0-21swapper/311:35:023
144391699403,5cyclictest131rcu_preempt11:30:003
1443916994030,6cyclictest306-21in:imuxsock09:10:013
1443916994029,8cyclictest306-21in:imuxsock12:10:003
144391699402,7cyclictest0-21swapper/308:10:023
1443916994026,11cyclictest37-21ksoftirqd/308:00:013
144391699402,5cyclictest131rcu_preempt10:00:003
1443916994023,12cyclictest235-21systemd-journal09:35:003
144391699401,6cyclictest131rcu_preempt12:20:003
144391699401,5cyclictest131rcu_preempt11:25:023
144391699401,4cyclictest131rcu_preempt07:54:593
1443916994014,20cyclictest131rcu_preempt11:05:013
1443916994014,20cyclictest131rcu_preempt11:05:013
1443912994030,8cyclictest30-21ksoftirqd/208:10:022
1443912994029,8cyclictest1457521-21systemd-run11:35:012
1443912994029,8cyclictest1457521-21systemd-run11:35:012
1443912994028,9cyclictest1457932-21sed11:40:022
1443912994027,9cyclictest1448977-21cron08:50:002
144391299401,8cyclictest0-21swapper/208:36:362
1443912994014,17cyclictest1452346-21kworker/u8:209:55:022
144390799406,18cyclictest131rcu_preempt08:10:021
1443907994026,11cyclictest23-21ksoftirqd/111:05:011
1443907994026,11cyclictest23-21ksoftirqd/111:05:011
144391699398,9cyclictest37-21ksoftirqd/307:20:213
144391699392,5cyclictest131rcu_preempt11:40:013
1443916993920,14cyclictest235-21systemd-journal10:15:003
144391699391,6cyclictest131rcu_preempt07:20:003
144391699391,6cyclictest0-21swapper/307:30:013
144391699391,5cyclictest131rcu_preempt09:55:023
1443912993927,7cyclictest30-21ksoftirqd/207:45:002
1443912993924,11cyclictest30-21ksoftirqd/212:17:002
1443912993914,21cyclictest291rcuc/211:55:022
1443907993929,6cyclictest306-21in:imuxsock09:30:001
1443907993927,8cyclictest23-21ksoftirqd/107:20:021
1443907993925,10cyclictest23-21ksoftirqd/111:35:021
1443907993925,10cyclictest23-21ksoftirqd/111:35:021
144390799391,34cyclictest23-21ksoftirqd/109:25:021
144390799391,34cyclictest23-21ksoftirqd/109:25:021
1443916993829,5cyclictest306-21in:imuxsock12:00:013
1443916993827,7cyclictest306-21in:imuxsock12:40:003
144391699382,5cyclictest131rcu_preempt09:50:003
144391699381,3cyclictest131rcu_preempt10:25:023
1443912993822,12cyclictest235-21systemd-journal12:35:012
1443912993822,11cyclictest30-21ksoftirqd/208:20:002
1443912993822,11cyclictest1454687-21run-parts10:39:592
1443912993821,12cyclictest272-21dbus-daemon07:55:012
1443912993821,10cyclictest1459548-21modprobe12:15:002
1443912993820,13cyclictest272-21dbus-daemon09:50:012
144391299381,9cyclictest30-21ksoftirqd/210:20:002
144391299381,9cyclictest30-21ksoftirqd/210:20:002
1443912993818,5cyclictest30-21ksoftirqd/208:00:002
144391299381,33cyclictest30-21ksoftirqd/212:25:022
1443907993828,8cyclictest23-21ksoftirqd/109:05:001
1443907993827,8cyclictest23-21ksoftirqd/112:10:001
1443907993826,9cyclictest23-21ksoftirqd/109:20:021
1443907993826,8cyclictest235-21systemd-journal07:57:561
1443907993826,7cyclictest1454433-21munin-run10:35:011
1443907993825,10cyclictest272-21dbus-daemon12:00:011
144390799382,4cyclictest131rcu_preempt07:14:591
1443907993823,11cyclictest23-21ksoftirqd/110:20:021
1443907993823,11cyclictest23-21ksoftirqd/110:20:021
1443907993819,14cyclictest235-21systemd-journal09:55:011
1443916993728,5cyclictest306-21in:imuxsock08:50:013
144391699372,5cyclictest131rcu_preempt07:15:003
1443916993722,11cyclictest1452915-21systemctl10:05:013
144391699371,5cyclictest131rcu_preempt10:20:003
144391699371,5cyclictest131rcu_preempt10:20:003
144391699371,5cyclictest131rcu_preempt09:45:013
144391299375,6cyclictest131rcu_preempt09:05:002
144391299373,5cyclictest131rcu_preempt10:29:362
1443912993721,12cyclictest1455689-21cron11:00:002
144391299371,5cyclictest131rcu_preempt12:05:022
144391299371,5cyclictest131rcu_preempt12:05:022
1443912993710,22cyclictest1451365-21unin-run09:35:012
1443907993727,6cyclictest23-21ksoftirqd/108:20:021
1443907993727,5cyclictest306-21in:imuxsock11:15:011
1443907993725,9cyclictest235-21systemd-journal09:50:001
1443907993716,16cyclictest235-21systemd-journal12:15:001
1443916993624,8cyclictest235-21systemd-journal11:12:363
1443916993624,7cyclictest37-21ksoftirqd/312:35:013
1443916993619,12cyclictest235-21systemd-journal08:51:363
1443912993626,8cyclictest30-21ksoftirqd/209:40:022
1443912993626,8cyclictest30-21ksoftirqd/209:40:012
1443912993621,11cyclictest1445399-21tr07:40:002
1443912993620,11cyclictest0-21swapper/210:55:012
144390799363,3cyclictest131rcu_preempt10:45:021
1443907993625,9cyclictest23-21ksoftirqd/107:35:011
1443907993625,9cyclictest23-21ksoftirqd/107:35:011
1443907993625,7cyclictest1447719-21modprobe08:25:001
144390799362,3cyclictest131rcu_preempt07:49:591
1443907993623,10cyclictest23-21ksoftirqd/109:45:001
1443907993622,9cyclictest272-21dbus-daemon07:45:011
144390799362,28cyclictest1448475-21systemctl08:40:001
1443907993620,12cyclictest1-21systemd07:40:011
144390799361,3cyclictest131rcu_preempt08:55:021
1443916993523,9cyclictest37-21ksoftirqd/309:25:033
144391699351,5cyclictest131rcu_preempt08:05:013
144391699351,3cyclictest131rcu_preempt08:35:023
144391299355,6cyclictest131rcu_preempt09:15:012
1443912993524,8cyclictest1453909-21sshd10:22:362
1443912993524,8cyclictest1451075-21sshd09:26:362
1443912993523,8cyclictest1460548-21sed12:30:022
144391299351,5cyclictest131rcu_preempt09:20:022
144391299351,19cyclictest30-21ksoftirqd/208:45:012
1443907993527,6cyclictest23-21ksoftirqd/108:35:001
1443907993523,9cyclictest23-21ksoftirqd/109:10:021
1443907993523,9cyclictest1457232-21sshd11:26:361
1443907993522,9cyclictest306-21in:imuxsock11:50:001
1443907993522,9cyclictest306-21in:imuxsock11:50:001
144391699344,5cyclictest131rcu_preempt09:10:363
1443916993423,9cyclictest37-21ksoftirqd/307:35:023
1443916993423,9cyclictest37-21ksoftirqd/307:35:023
1443916993423,8cyclictest37-21ksoftirqd/308:42:363
144391699341,4cyclictest131rcu_preempt07:50:003
1443912993427,4cyclictest1454431-21munin-plugin-st10:35:012
1443912993424,7cyclictest30-21ksoftirqd/208:15:032
1443912993422,10cyclictest30-21ksoftirqd/207:20:022
1443912993421,10cyclictest1-21systemd11:15:012
144391299341,5cyclictest131rcu_preempt11:20:032
144390799344,3cyclictest131rcu_preempt09:40:001
144390799344,3cyclictest131rcu_preempt09:40:001
144390799343,3cyclictest131rcu_preempt11:00:001
1443907993424,7cyclictest1446693-21systemd-run08:05:001
1443907993421,10cyclictest23-21ksoftirqd/108:56:201
144391699333,4cyclictest131rcu_preempt07:10:013
1443916993324,7cyclictest37-21ksoftirqd/309:25:023
144391299337,6cyclictest131rcu_preempt11:45:002
1443912993325,5cyclictest1458973-21latency_hist12:00:022
1443912993323,7cyclictest30-21ksoftirqd/210:15:012
1443912993319,11cyclictest272-21dbus-daemon08:55:512
1443907993325,5cyclictest1450319-21modprobe09:14:591
1443907993323,7cyclictest23-21ksoftirqd/111:55:021
144391299323,21cyclictest1453152-21modprobe10:10:002
144391299322,5cyclictest131rcu_preempt07:15:022
1443912993225,5cyclictest271-21cron08:05:002
1443912993222,7cyclictest1447858-21grep08:25:012
1443912993219,9cyclictest0-21swapper/209:45:002
1443907993226,4cyclictest1448986-21latency_hist08:50:001
1443907993222,8cyclictest23-21ksoftirqd/112:25:011
1443907993222,7cyclictest23-21ksoftirqd/110:05:011
1443907993220,6cyclictest306-21in:imuxsock10:50:011
1443907993220,6cyclictest306-21in:imuxsock10:50:011
1443912993119,8cyclictest306-21in:imuxsock07:35:022
1443912993119,8cyclictest306-21in:imuxsock07:35:022
144390799313,3cyclictest131rcu_preempt10:51:371
144390799313,3cyclictest131rcu_preempt08:40:031
1443907993125,4cyclictest1457006-21latency_hist11:25:011
144391299302,5cyclictest0-21swapper/211:30:012
1443912993022,5cyclictest1443844-21latency_hist07:10:012
1443912993020,8cyclictest30-21ksoftirqd/209:25:012
1443912993020,8cyclictest30-21ksoftirqd/209:25:012
1443912993018,10cyclictest30-21ksoftirqd/208:29:362
1443912993018,10cyclictest30-21ksoftirqd/208:29:362
1443907993016,10cyclictest1453338-21cut10:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional