You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-03 - 04:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sat Jan 03, 2026 00:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2644969991011,2cyclictest0-21swapper/022:10:020
2644969997965,9cyclictest12-21ksoftirqd/021:20:020
2644969997665,4cyclictest0-21swapper/021:40:020
2644969997167,2cyclictest0-21swapper/023:45:020
264496999692,15cyclictest2656052-21munin-run22:45:010
264497399685,7cyclictest23-21ksoftirqd/100:00:011
264496999682,15cyclictest2660157-21sshd00:03:340
2644969996815,10cyclictest131rcu_preempt20:05:020
264496999672,5cyclictest2658903-21systemctl23:40:020
264496999672,5cyclictest2658903-21systemctl23:40:010
264496999662,4cyclictest131rcu_preempt00:20:020
264496999662,19cyclictest2645143-21sshd19:14:400
2644969996558,5cyclictest19350irq/34-mmc100:10:020
2644969996530,30cyclictest2654732-21sendmail22:20:010
264496999651,60cyclictest0-21swapper/020:38:160
264496999642,15cyclictest2656291-21munin-run22:50:000
264496999642,15cyclictest2655509-21munin-run22:34:590
2644973996314,13cyclictest23-21ksoftirqd/123:30:001
2644969996336,18cyclictest131rcu_preempt21:50:020
264496999631,13cyclictest2652384-21sshd21:32:390
2644973996219,10cyclictest23-21ksoftirqd/122:20:001
264496999624,23cyclictest2659884-40mandb00:00:010
264496999622,9cyclictest2657072-21munin-run23:05:010
264496999622,9cyclictest2657072-21munin-run23:05:000
2644969996224,35cyclictest12-21ksoftirqd/022:25:000
2644969996224,34cyclictest2660875-21latency_hist00:15:020
2644969996224,34cyclictest2660875-21latency_hist00:15:020
264496999622,17cyclictest2645430-21munin-run19:20:000
264496999622,12cyclictest2656541-21munin-run22:55:010
264496999622,11cyclictest2650344-21munin-run20:55:010
264496999613,16cyclictest2649019-21sshd20:26:380
264496999613,12cyclictest2651365-21modprobe21:15:010
264496999612,54cyclictest2645064-21sed19:10:020
2644969996114,42cyclictest2656801-21sendmail23:00:000
264496999611,13cyclictest2658340-21sshd23:28:390
2644980996016,13cyclictest37-21ksoftirqd/322:50:003
264496999602,8cyclictest2646214-21munin-run19:35:000
264496999602,14cyclictest2661987-21modprobe00:40:000
264496999602,13cyclictest2653453-21munin-run21:55:020
264496999602,13cyclictest2653453-21munin-run21:55:010
264496999601,15cyclictest2647454-21sshd19:56:400
264496999601,13cyclictest2657297-21sshd23:06:590
264496999592,17cyclictest2658106-21latency_hist23:25:000
264496999592,12cyclictest2646730-21munin-run19:45:000
264496999591,14cyclictest2645914-21sshd19:25:400
264496999591,14cyclictest2645914-21sshd19:25:390
264496999591,12cyclictest2649804-21sshd20:42:390
264496999591,12cyclictest2648527-21munin-run20:20:000
2644973995812,9cyclictest23-21ksoftirqd/120:05:021
264496999582,16cyclictest2661210-21cron00:25:000
264496999582,16cyclictest2652334-21cat21:30:020
2644969995821,33cyclictest12-21ksoftirqd/000:35:010
264496999581,2cyclictest0-21swapper/020:05:020
264496999581,12cyclictest364-21snmpd23:17:180
264496999581,12cyclictest2651046-21cut21:05:010
264496999581,10cyclictest364-21snmpd22:29:020
264496999581,10cyclictest364-21snmpd22:29:010
264497399576,8cyclictest23-21ksoftirqd/122:30:021
264497399576,8cyclictest23-21ksoftirqd/122:30:021
264496999572,13cyclictest2648274-21munin-run20:15:010
264496999572,12cyclictest2646438-21cron19:40:000
264498099568,8cyclictest37-21ksoftirqd/321:25:003
264498099564,10cyclictest37-21ksoftirqd/320:00:013
2644980995611,11cyclictest37-21ksoftirqd/300:00:013
264496999564,7cyclictest2649400-21latency_hist20:35:010
264496999562,7cyclictest2651092-21sshd21:08:390
264496999561,14cyclictest2653948-21sshd22:03:400
264496999561,11cyclictest2651908-21munin-run21:25:000
2644969995533,19cyclictest2661474-21latency_hist00:30:030
264496999552,8cyclictest2645685-21cron19:25:000
264496999552,10cyclictest2658712-21grep23:35:020
264496999551,11cyclictest2652933-21cron21:45:000
264498099547,11cyclictest37-21ksoftirqd/319:30:033
264498099547,11cyclictest37-21ksoftirqd/319:30:023
2644980995416,9cyclictest37-21ksoftirqd/320:20:013
2644978995434,14cyclictest275-21dbus-daemon20:00:002
264497399545,8cyclictest23-21ksoftirqd/121:10:011
2644973995436,12cyclictest238-21systemd-journal22:54:591
264496999542,12cyclictest2659643-21munin-run23:55:010
264496999541,13cyclictest364-21snmpd21:58:510
264498099536,10cyclictest37-21ksoftirqd/320:40:023
2644978995322,25cyclictest275-21dbus-daemon19:20:012
2644973995336,12cyclictest275-21dbus-daemon22:10:011
264496999533,9cyclictest2650568-21cron21:00:010
264496999533,9cyclictest2650568-21cron21:00:000
264496999532,7cyclictest2655747-21cron22:40:010
264496999532,7cyclictest2648801-21grep20:25:000
264496999532,7cyclictest2648801-21grep20:25:000
264498099525,12cyclictest37-21ksoftirqd/319:40:293
264497399524,7cyclictest23-21ksoftirqd/123:35:011
264496999523,14cyclictest1-21systemd22:15:010
264498099514,9cyclictest37-21ksoftirqd/319:25:003
2644978995136,10cyclictest238-21systemd-journal21:20:012
264497899512,5cyclictest131rcu_preempt22:00:002
2644973995138,9cyclictest221rcuc/120:25:011
2644973995138,9cyclictest221rcuc/120:25:011
2644973995135,12cyclictest1-21systemd22:45:001
264497399512,14cyclictest23-21ksoftirqd/100:10:021
264496999514,7cyclictest2659395-21cut23:49:590
264496999512,9cyclictest2650083-21munin-plugin-st20:50:010
264496999512,9cyclictest2646980-21munin-plugin-st19:50:010
264496999512,9cyclictest2646980-21munin-plugin-st19:50:000
2644969995121,26cyclictest12-21ksoftirqd/023:15:000
264496999511,2cyclictest0-21swapper/019:55:010
2644980995032,13cyclictest2658849-21cron23:40:013
2644980995032,13cyclictest2658849-21cron23:40:003
2644978995034,9cyclictest309-21in:imuxsock22:40:002
2644978995033,11cyclictest1-21systemd00:20:002
2644978995031,13cyclictest238-21systemd-journal23:35:012
264497399505,13cyclictest23-21ksoftirqd/119:15:011
2644973995015,11cyclictest23-21ksoftirqd/121:15:021
2644973995010,9cyclictest23-21ksoftirqd/100:05:021
264498099496,6cyclictest37-21ksoftirqd/321:05:003
264498099494,9cyclictest37-21ksoftirqd/320:05:013
2644980994930,14cyclictest238-21systemd-journal21:55:013
2644980994930,14cyclictest238-21systemd-journal21:55:003
2644978994935,8cyclictest309-21in:imuxsock00:35:022
264497399496,15cyclictest23-21ksoftirqd/120:35:391
264497399495,10cyclictest23-21ksoftirqd/121:05:011
264497399494,29cyclictest221rcuc/122:30:031
264497399491,12cyclictest23-21ksoftirqd/121:55:001
264497399491,12cyclictest23-21ksoftirqd/121:55:001
264498099485,8cyclictest37-21ksoftirqd/321:20:013
264498099485,7cyclictest37-21ksoftirqd/323:10:003
264498099485,7cyclictest37-21ksoftirqd/320:15:013
264498099484,9cyclictest37-21ksoftirqd/300:30:013
264498099484,11cyclictest37-21ksoftirqd/323:45:003
2644973994835,10cyclictest23-21ksoftirqd/119:45:031
264497399482,9cyclictest23-21ksoftirqd/100:40:011
264498099479,10cyclictest37-21ksoftirqd/300:15:013
264498099479,10cyclictest37-21ksoftirqd/300:15:013
264498099471,12cyclictest37-21ksoftirqd/320:50:023
2644978994729,13cyclictest275-21dbus-daemon21:35:012
264497399479,9cyclictest23-21ksoftirqd/123:55:031
264497399476,15cyclictest23-21ksoftirqd/123:40:011
264497399476,15cyclictest23-21ksoftirqd/123:40:011
264497399475,8cyclictest23-21ksoftirqd/119:40:021
264497399472,12cyclictest23-21ksoftirqd/119:20:001
2644973994712,5cyclictest131rcu_preempt20:10:031
264498099468,4cyclictest131rcu_preempt23:12:383
264498099465,9cyclictest37-21ksoftirqd/323:55:013
264498099464,9cyclictest37-21ksoftirqd/322:34:593
264498099464,12cyclictest37-21ksoftirqd/321:45:003
2644980994624,16cyclictest238-21systemd-journal21:15:013
264498099461,11cyclictest37-21ksoftirqd/320:25:013
264498099461,11cyclictest37-21ksoftirqd/320:25:013
264497899468,20cyclictest2653661-21kworker/u8:022:15:022
264497899464,4cyclictest131rcu_preempt21:55:012
264497899464,4cyclictest131rcu_preempt21:55:002
2644978994614,6cyclictest131rcu_preempt22:20:002
264497399469,9cyclictest23-21ksoftirqd/123:00:021
264497399465,6cyclictest23-21ksoftirqd/123:20:011
264497399465,6cyclictest23-21ksoftirqd/121:40:011
2644973994636,6cyclictest23-21ksoftirqd/100:15:031
2644973994636,6cyclictest23-21ksoftirqd/100:15:021
264498099458,4cyclictest131rcu_preempt22:55:013
264498099455,7cyclictest37-21ksoftirqd/322:45:003
264498099454,7cyclictest37-21ksoftirqd/323:33:393
264498099453,7cyclictest37-21ksoftirqd/321:00:033
264498099453,7cyclictest37-21ksoftirqd/321:00:023
2644980994526,14cyclictest238-21systemd-journal20:55:003
2644980994519,8cyclictest37-21ksoftirqd/321:57:183
2644978994531,9cyclictest309-21in:imuxsock20:20:002
2644978994527,13cyclictest30-21ksoftirqd/222:10:012
2644973994530,11cyclictest275-21dbus-daemon21:50:011
2644973994530,11cyclictest275-21dbus-daemon19:25:011
264497399452,9cyclictest23-21ksoftirqd/121:20:021
264497399452,8cyclictest23-21ksoftirqd/119:35:011
264497399452,7cyclictest23-21ksoftirqd/121:25:011
264498099446,8cyclictest37-21ksoftirqd/323:46:383
264498099446,7cyclictest37-21ksoftirqd/322:10:023
264498099445,6cyclictest131rcu_preempt22:56:393
264498099444,8cyclictest37-21ksoftirqd/319:40:003
264497399447,10cyclictest23-21ksoftirqd/100:33:391
264497399446,7cyclictest23-21ksoftirqd/120:55:021
264497399446,7cyclictest23-21ksoftirqd/120:35:001
264497399445,8cyclictest23-21ksoftirqd/121:30:011
264497399445,7cyclictest23-21ksoftirqd/122:05:011
264497399444,5cyclictest23-21ksoftirqd/120:00:001
264497399444,11cyclictest23-21ksoftirqd/121:33:391
2644973994429,11cyclictest275-21dbus-daemon21:45:001
264497399441,13cyclictest2648601-21sed20:20:011
264498099438,7cyclictest0-21swapper/321:06:463
264498099437,8cyclictest37-21ksoftirqd/320:45:013
264498099435,9cyclictest37-21ksoftirqd/319:15:023
264498099435,8cyclictest37-21ksoftirqd/320:27:383
264497899433,3cyclictest131rcu_preempt21:50:012
2644978994327,11cyclictest275-21dbus-daemon21:30:002
2644978994326,10cyclictest238-21systemd-journal23:20:002
2644978994318,18cyclictest275-21dbus-daemon21:15:002
264497399434,10cyclictest23-21ksoftirqd/122:00:011
264497399431,4cyclictest0-21swapper/123:44:591
264498099426,6cyclictest37-21ksoftirqd/321:30:023
264498099426,10cyclictest37-21ksoftirqd/320:06:393
264498099423,5cyclictest131rcu_preempt21:45:393
2644978994227,12cyclictest2656896-21sed23:00:022
2644978994220,14cyclictest275-21dbus-daemon21:05:012
264497399426,10cyclictest23-21ksoftirqd/120:43:401
264497399425,9cyclictest23-21ksoftirqd/122:50:021
264497399421,9cyclictest23-21ksoftirqd/123:25:001
264497399421,8cyclictest2659396-21modprobe23:50:011
2644973994212,8cyclictest23-21ksoftirqd/100:30:011
264498099415,7cyclictest37-21ksoftirqd/319:31:393
2644978994121,5cyclictest30-21ksoftirqd/221:39:592
264497399418,5cyclictest0-21swapper/123:08:391
264497399415,6cyclictest23-21ksoftirqd/119:50:031
264497399414,7cyclictest23-21ksoftirqd/121:00:031
264497399414,7cyclictest23-21ksoftirqd/121:00:021
264497399412,8cyclictest23-21ksoftirqd/100:25:011
2644973994125,12cyclictest238-21systemd-journal22:25:011
264497399411,8cyclictest23-21ksoftirqd/122:15:021
264497399411,8cyclictest23-21ksoftirqd/120:30:001
2644973994113,5cyclictest131rcu_preempt19:50:011
2644973994113,5cyclictest131rcu_preempt19:50:011
264498099404,6cyclictest37-21ksoftirqd/319:50:033
264498099404,6cyclictest37-21ksoftirqd/319:50:023
2644980994024,11cyclictest275-21dbus-daemon22:40:013
264498099402,12cyclictest37-21ksoftirqd/322:30:013
264498099402,12cyclictest37-21ksoftirqd/322:30:003
2644978994029,8cyclictest2647724-21sshd20:00:402
2644978994027,10cyclictest2647428-21sed19:55:022
2644978994026,10cyclictest30-21ksoftirqd/200:30:012
2644978994023,11cyclictest0-21swapper/222:45:032
2644978994022,12cyclictest275-21dbus-daemon20:15:002
264497399404,8cyclictest23-21ksoftirqd/120:50:001
264497399404,10cyclictest23-21ksoftirqd/123:01:591
264497399404,10cyclictest23-21ksoftirqd/123:01:581
264498099395,8cyclictest37-21ksoftirqd/323:30:013
264498099395,7cyclictest37-21ksoftirqd/322:01:393
264498099394,9cyclictest37-21ksoftirqd/300:40:023
264498099393,7cyclictest37-21ksoftirqd/300:20:383
264498099392,8cyclictest37-21ksoftirqd/319:20:003
2644980993910,5cyclictest131rcu_preempt20:30:403
2644980993910,5cyclictest131rcu_preempt00:16:403
264497899395,4cyclictest131rcu_preempt19:15:002
2644978993929,8cyclictest30-21ksoftirqd/220:25:012
2644978993929,8cyclictest30-21ksoftirqd/220:25:012
2644978993929,5cyclictest2654106-21latency_hist22:05:012
2644978993925,8cyclictest309-21in:imuxsock23:05:012
2644978993925,8cyclictest309-21in:imuxsock23:05:012
2644978993925,10cyclictest1-21systemd23:30:002
2644978993924,11cyclictest30-21ksoftirqd/223:55:032
264498099384,8cyclictest37-21ksoftirqd/321:30:493
2644980993825,9cyclictest2660396-21sshd00:05:403
2644980993823,10cyclictest1-21systemd22:20:013
2644978993828,7cyclictest30-21ksoftirqd/221:25:022
2644978993827,7cyclictest2657571-21munin-plugin-st23:15:002
2644978993822,10cyclictest309-21in:imuxsock22:55:002
2644978993820,12cyclictest2659134-21modprobe23:45:002
264497399383,6cyclictest23-21ksoftirqd/119:10:031
2644973993827,6cyclictest309-21in:imuxsock23:15:011
2644973993826,9cyclictest2655739-21sshd22:38:391
264497399382,10cyclictest23-21ksoftirqd/120:15:011
264498099373,6cyclictest37-21ksoftirqd/323:05:013
264498099373,6cyclictest37-21ksoftirqd/323:05:003
264497899375,22cyclictest2645700-21munin-plugin-st19:25:002
2644978993725,7cyclictest309-21in:imuxsock21:45:012
2644978993724,8cyclictest1-21systemd20:10:012
2644978993721,11cyclictest0-21swapper/223:50:002
264497899372,10cyclictest2656349-21latency_hist22:50:022
2644978993720,11cyclictest275-21dbus-daemon23:10:012
2644978993719,14cyclictest275-21dbus-daemon20:35:002
2644978993713,18cyclictest275-21dbus-daemon19:40:002
2644973993725,9cyclictest23-21ksoftirqd/100:20:011
264497399371,7cyclictest23-21ksoftirqd/119:30:021
264497399371,7cyclictest23-21ksoftirqd/119:30:021
264498099366,3cyclictest131rcu_preempt19:54:393
2644980993629,4cyclictest2655005-21latency_hist22:25:003
2644980993621,11cyclictest275-21dbus-daemon00:35:013
2644978993628,6cyclictest2660175-21sh00:04:592
2644978993626,6cyclictest2659641-21munin-plugin-st23:55:002
2644978993625,7cyclictest30-21ksoftirqd/219:50:022
2644978993625,7cyclictest30-21ksoftirqd/219:50:012
2644978993624,9cyclictest30-21ksoftirqd/219:45:012
2644978993616,14cyclictest275-21dbus-daemon20:45:012
264497899361,3cyclictest131rcu_preempt21:10:002
264498099356,5cyclictest131rcu_preempt22:05:553
2644978993518,13cyclictest275-21dbus-daemon23:40:012
2644978993518,13cyclictest275-21dbus-daemon23:40:002
2644978993512,19cyclictest1-21systemd21:00:012
2644978993512,19cyclictest1-21systemd21:00:012
264498099341,9cyclictest37-21ksoftirqd/300:05:003
264497899345,5cyclictest131rcu_preempt20:50:012
264497899342,4cyclictest131rcu_preempt22:25:022
264497899341,3cyclictest131rcu_preempt19:30:032
264497899341,3cyclictest131rcu_preempt19:30:022
264498099338,4cyclictest131rcu_preempt23:17:023
2644980993321,9cyclictest309-21in:imuxsock23:20:393
264497899338,21cyclictest291rcuc/220:40:012
264497899334,3cyclictest131rcu_preempt22:34:592
2644978993326,5cyclictest30-21ksoftirqd/200:25:022
2644978993325,5cyclictest2660889-21latency_hist00:15:022
264497899332,4cyclictest131rcu_preempt23:20:402
2644978993324,6cyclictest2655255-21latency_hist22:30:022
2644978993324,6cyclictest2655255-21latency_hist22:30:012
2644978993318,10cyclictest0-21swapper/200:39:592
264497899331,3cyclictest131rcu_preempt00:10:012
2644980993225,4cyclictest2652782-21latency_hist21:40:023
264497899324,21cyclictest2649227-21grep20:30:012
264497899322,4cyclictest131rcu_preempt19:10:022
2644978993219,10cyclictest275-21dbus-daemon20:55:022
264497899312,3cyclictest131rcu_preempt00:12:392
2644978993018,8cyclictest30-21ksoftirqd/219:34:592
2644980992817,8cyclictest37-21ksoftirqd/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional