You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-09 - 22:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Fri Jan 09, 2026 12:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3452239991091,3cyclictest131rcu_preempt08:20:023
3452239999758,30cyclictest37-21ksoftirqd/311:01:443
345223399791,8cyclictest131rcu_preempt10:59:551
3452227997867,6cyclictest3452226-21latency_hist07:10:020
3452233997444,7cyclictest23-21ksoftirqd/110:30:021
3452227997412,52cyclictest12-21ksoftirqd/010:59:200
3452239997226,5cyclictest3468545-21mandb10:58:443
3452239997224,6cyclictest309-21in:imuxsock10:19:593
3452227997268,2cyclictest0-21swapper/012:10:020
3452227997222,7cyclictest12-21ksoftirqd/008:20:020
3452227996963,4cyclictest151rcuc/010:45:020
3452239996722,13cyclictest37-21ksoftirqd/310:45:123
3452227996637,20cyclictest12-21ksoftirqd/011:00:110
3452227996614,42cyclictest12-21ksoftirqd/010:50:530
3452239996330,14cyclictest37-21ksoftirqd/310:37:023
3452239996330,14cyclictest37-21ksoftirqd/310:37:023
3452239996329,5cyclictest3454748-21munin-plugin-st08:00:003
3452227996246,13cyclictest0-21swapper/010:45:030
3452239996121,11cyclictest275-21dbus-daemon12:15:013
3452233996127,24cyclictest131rcu_preempt11:00:501
3452239996022,11cyclictest37-21ksoftirqd/310:44:533
345222799589,4cyclictest12-21ksoftirqd/012:05:000
3452227995823,31cyclictest3463185-21apt10:36:050
3452227995823,31cyclictest3463185-21apt10:36:050
345222799575,15cyclictest3481476-21modprobe12:20:010
3452239995623,10cyclictest3462370-21sed10:25:023
3452239995621,11cyclictest37-21ksoftirqd/307:30:003
3452227995539,12cyclictest3467602-21systemd-journal12:39:590
3452239995418,12cyclictest37-21ksoftirqd/310:34:493
345223999535,34cyclictest3476677-21apt-get11:05:143
345222799538,10cyclictest12-21ksoftirqd/008:15:000
3452227995335,13cyclictest3467602-21systemd-journal11:59:590
345222799523,10cyclictest3477356-21systemd-run-gen11:05:200
3452235995134,13cyclictest238-21systemd-journal10:17:002
3452227995112,36cyclictest3460976-21latency_hist10:00:010
3452239995036,9cyclictest3453717-21sh07:39:593
3452239995013,7cyclictest131rcu_preempt08:50:003
3452235995036,10cyclictest1-21systemd08:50:012
3452235995034,12cyclictest1-21systemd12:15:002
3452235995033,13cyclictest238-21systemd-journal10:15:002
345223399504,6cyclictest131rcu_preempt10:45:501
3452233995033,13cyclictest1-21systemd09:00:001
345223599493,3cyclictest131rcu_preempt10:46:172
345223399499,6cyclictest131rcu_preempt10:55:001
3452233994935,10cyclictest275-21dbus-daemon10:00:001
3452227994915,28cyclictest309-21in:imuxsock09:09:590
3452235994833,12cyclictest238-21systemd-journal09:15:032
345222799485,9cyclictest3478382-21munin-run11:20:010
3452239994736,6cyclictest3479411-21sendmail11:40:003
3452235994733,10cyclictest3467522-21apt-config10:57:252
3452233994731,12cyclictest23-21ksoftirqd/112:20:001
3452233994729,14cyclictest3452668-21cron07:20:001
3452233994727,15cyclictest1-21systemd11:05:121
3452227994730,13cyclictest3467602-21systemd-journal12:31:550
3452227994730,13cyclictest3467602-21systemd-journal12:31:550
3452227994728,15cyclictest3461232-21sshd10:04:520
3452235994630,12cyclictest1-21systemd12:05:002
3452233994631,11cyclictest23-21ksoftirqd/110:38:551
3452233994631,11cyclictest23-21ksoftirqd/110:38:541
3452233994630,11cyclictest3467602-21systemd-journal11:54:591
345223399462,7cyclictest131rcu_preempt10:34:481
3452233994610,26cyclictest221rcuc/107:40:011
3452227994634,7cyclictest3452722-21unin-run07:20:010
3452227994623,5cyclictest12-21ksoftirqd/009:15:010
3452227994622,4cyclictest12-21ksoftirqd/007:35:030
3452227994622,4cyclictest12-21ksoftirqd/007:35:020
345222799462,12cyclictest3462013-21sendmail10:19:590
3452239994528,13cyclictest275-21dbus-daemon09:05:013
3452239994528,13cyclictest275-21dbus-daemon09:05:003
3452235994527,13cyclictest30-21ksoftirqd/211:00:112
345223399454,6cyclictest131rcu_preempt11:15:021
345223399452,8cyclictest23-21ksoftirqd/109:25:011
345222799457,24cyclictest3459930-21modprobe09:40:000
345222799453,4cyclictest131rcu_preempt08:10:030
345222799453,4cyclictest131rcu_preempt08:10:030
3452227994527,12cyclictest275-21dbus-daemon08:05:000
345223999443,3cyclictest131rcu_preempt12:20:013
3452235994432,8cyclictest3457234-21cat08:45:032
3452235994432,8cyclictest3457234-21cat08:45:022
3452235994431,10cyclictest30-21ksoftirqd/207:54:532
3452235994429,12cyclictest3467602-21systemd-journal11:16:542
3452233994429,11cyclictest3458365-21cron09:10:001
3452227994437,4cyclictest16950irq/224-800000007:25:010
3452227994427,12cyclictest1-21systemd08:00:010
345222799442,13cyclictest3453794-21sed07:40:010
3452235994333,8cyclictest30-21ksoftirqd/210:44:592
3452235994327,12cyclictest3463866-21dpkg10:39:262
3452235994327,12cyclictest3463866-21dpkg10:39:252
3452235994327,12cyclictest30-21ksoftirqd/210:30:032
3452233994325,14cyclictest23-21ksoftirqd/110:43:391
345222799437,25cyclictest3454496-21cut07:55:000
345222799433,9cyclictest3458931-21sendmail-msp09:20:010
3452227994327,7cyclictest309-21in:imuxsock09:00:010
3452227994325,5cyclictest12-21ksoftirqd/007:15:010
3452239994227,11cyclictest3459927-21cron09:40:003
3452239994222,15cyclictest3452668-21cron07:20:003
3452235994231,9cyclictest30-21ksoftirqd/207:30:022
345223399423,3cyclictest131rcu_preempt07:55:011
3452233994229,9cyclictest309-21in:imuxsock11:55:021
3452233994226,12cyclictest1-21systemd08:20:001
3452233994214,6cyclictest131rcu_preempt12:35:011
3452233994214,6cyclictest131rcu_preempt12:35:001
3452233994211,5cyclictest131rcu_preempt11:21:551
3452227994225,12cyclictest3467602-21systemd-journal11:40:000
3452227994225,12cyclictest3462263-21sshd10:28:250
3452239994129,10cyclictest37-21ksoftirqd/311:45:013
3452239994129,10cyclictest37-21ksoftirqd/307:44:553
3452239994125,13cyclictest3482689-21sed12:40:013
3452235994132,5cyclictest309-21in:imuxsock08:10:022
3452235994127,11cyclictest30-21ksoftirqd/207:25:022
3452235994127,10cyclictest309-21in:imuxsock11:10:022
345223399415,8cyclictest23-21ksoftirqd/110:22:311
3452233994131,7cyclictest0-21swapper/107:10:011
3452233994129,9cyclictest23-21ksoftirqd/109:30:031
3452233994129,8cyclictest1-21systemd10:10:011
3452233994127,10cyclictest3454743-21cron07:59:591
3452233994122,14cyclictest1-21systemd12:40:001
345223399411,8cyclictest311-21rs:main1
345223399411,8cyclictest311-21rs:main1
345222799413,6cyclictest3456824-21munin-run08:40:000
3452227994127,12cyclictest12-21ksoftirqd/011:15:010
3452227994124,13cyclictest238-21systemd-journal09:35:010
3452239994028,8cyclictest3461239-21cron10:04:593
3452239994028,8cyclictest1-21systemd09:10:003
3452239994023,13cyclictest37-21ksoftirqd/308:10:033
3452239994023,13cyclictest37-21ksoftirqd/308:10:023
3452235994026,11cyclictest275-21dbus-daemon10:22:302
3452235994025,11cyclictest3460981-21systemctl09:59:592
345223599402,24cyclictest291rcuc/211:25:022
3452235994021,13cyclictest30-21ksoftirqd/208:35:002
345223399403,6cyclictest131rcu_preempt09:35:001
345223399402,5cyclictest131rcu_preempt08:05:021
3452233994024,11cyclictest275-21dbus-daemon12:25:011
345223399401,3cyclictest131rcu_preempt12:15:021
3452233994010,14cyclictest23-21ksoftirqd/112:05:011
345222799405,27cyclictest3459575-21grep09:30:020
345222799402,31cyclictest3457300-21latency_hist08:45:030
3452227994023,12cyclictest12-21ksoftirqd/010:34:480
3452227994021,14cyclictest1-21systemd11:35:010
3452227994021,14cyclictest1-21systemd11:35:000
345223999394,4cyclictest131rcu_preempt11:25:023
3452239993929,6cyclictest3455022-21munin-plugin-st08:05:003
3452239993927,9cyclictest3482020-21systemd-run12:30:013
3452239993925,9cyclictest238-21systemd-journal09:35:013
3452239993919,15cyclictest275-21dbus-daemon12:05:003
3452235993923,12cyclictest3477397-21mandb11:05:232
3452235993923,11cyclictest275-21dbus-daemon10:55:012
345223599391,4cyclictest131rcu_preempt11:30:002
345223399393,5cyclictest131rcu_preempt08:49:431
3452233993927,9cyclictest3460820-21wc09:55:021
3452233993926,9cyclictest23-21ksoftirqd/109:20:021
3452233993922,11cyclictest275-21dbus-daemon10:14:591
345222799395,9cyclictest12-21ksoftirqd/009:25:010
3452227993928,8cyclictest12-21ksoftirqd/008:30:010
3452227993927,8cyclictest3457595-21sh08:55:000
3452227993924,10cyclictest275-21dbus-daemon12:30:010
3452227993923,11cyclictest1-21systemd08:45:010
3452227993921,5cyclictest12-21ksoftirqd/009:50:010
3452227993921,15cyclictest0-21swapper/009:55:020
3452227993919,11cyclictest309-21in:imuxsock09:05:010
3452227993919,11cyclictest309-21in:imuxsock09:05:000
345223999385,9cyclictest37-21ksoftirqd/307:54:553
3452239993826,7cyclictest309-21in:imuxsock11:50:013
3452239993824,9cyclictest1-21systemd09:20:013
3452239993824,11cyclictest37-21ksoftirqd/308:30:013
3452239993822,12cyclictest1-21systemd08:40:003
345223999381,3cyclictest131rcu_preempt12:10:023
345223599383,19cyclictest3455275-21grep08:10:012
345223599383,19cyclictest3455275-21grep08:10:002
3452235993825,9cyclictest3453911-21sed07:40:012
3452235993825,10cyclictest1-21systemd12:40:012
345223399382,5cyclictest131rcu_preempt11:39:591
3452233993825,10cyclictest23-21ksoftirqd/109:45:011
3452233993824,9cyclictest23-21ksoftirqd/107:24:591
345222799387,5cyclictest131rcu_preempt11:25:000
345222799387,23cyclictest3462319-21modprobe10:25:010
345222799386,29cyclictest3457322-21sshd08:48:530
345222799383,32cyclictest3461489-21kworker/u8:110:10:010
3452227993829,5cyclictest3461891-21latency_hist10:15:020
3452227993823,10cyclictest3467602-21systemd-journal11:55:000
3452227993822,11cyclictest3467602-21systemd-journal11:50:010
3452227993821,13cyclictest1-21systemd08:35:010
345223999376,6cyclictest131rcu_preempt08:12:543
3452239993728,5cyclictest309-21in:imuxsock08:55:013
3452239993726,8cyclictest37-21ksoftirqd/307:34:303
3452239993726,8cyclictest37-21ksoftirqd/307:34:293
3452239993719,13cyclictest3467602-21systemd-journal11:20:013
3452239993719,13cyclictest275-21dbus-daemon11:59:593
3452239993718,13cyclictest275-21dbus-daemon07:50:003
3452235993727,8cyclictest30-21ksoftirqd/209:15:012
3452235993726,9cyclictest30-21ksoftirqd/209:30:012
3452235993725,9cyclictest3454991-21sshd08:01:542
3452235993725,10cyclictest30-21ksoftirqd/211:55:022
3452235993724,10cyclictest30-21ksoftirqd/212:35:012
3452235993724,10cyclictest30-21ksoftirqd/212:35:012
3452235993713,20cyclictest3456829-21systemctl08:40:012
3452233993729,5cyclictest3457084-21cron08:45:011
3452233993729,5cyclictest3457084-21cron08:45:001
3452233993728,6cyclictest3480969-21munin-run12:10:001
3452233993725,9cyclictest3453446-21sshd07:34:551
3452233993725,9cyclictest3453446-21sshd07:34:541
3452233993725,8cyclictest238-21systemd-journal09:40:011
345223399372,3cyclictest131rcu_preempt09:15:021
3452233993722,11cyclictest3460214-21cron09:45:001
3452227993720,13cyclictest3478916-21systemd-run11:30:020
345223999362,3cyclictest131rcu_preempt09:00:003
3452239993623,10cyclictest238-21systemd-journal10:00:013
345223999361,4cyclictest131rcu_preempt11:55:013
345223999361,14cyclictest3462615-21sh10:30:013
3452235993627,6cyclictest30-21ksoftirqd/212:20:022
3452235993626,5cyclictest309-21in:imuxsock12:10:012
3452235993620,12cyclictest275-21dbus-daemon08:55:002
345223399366,6cyclictest23-21ksoftirqd/108:29:541
345223399364,6cyclictest131rcu_preempt10:20:001
345223399364,28cyclictest221rcuc/107:30:031
3452233993627,5cyclictest309-21in:imuxsock08:25:001
3452233993625,8cyclictest3455489-21sshd08:10:551
3452233993621,11cyclictest275-21dbus-daemon09:05:021
3452233993621,11cyclictest275-21dbus-daemon09:05:011
3452233993621,11cyclictest275-21dbus-daemon08:10:011
3452233993621,11cyclictest275-21dbus-daemon08:10:001
345223399361,3cyclictest131rcu_preempt11:20:021
345223399361,3cyclictest131rcu_preempt08:35:011
345223399361,3cyclictest131rcu_preempt07:50:001
3452227993626,8cyclictest12-21ksoftirqd/012:15:010
3452227993624,9cyclictest12-21ksoftirqd/012:25:000
3452227993621,11cyclictest1-21systemd11:45:000
345222799361,31cyclictest0-21swapper/007:50:010
345222799361,10cyclictest12-21ksoftirqd/007:45:010
345223999353,3cyclictest131rcu_preempt07:10:023
3452239993528,3cyclictest37-21ksoftirqd/309:30:023
345223599355,6cyclictest131rcu_preempt10:10:002
3452235993523,9cyclictest3480620-21sed12:00:022
3452235993523,9cyclictest210-21jbd2/mmcblk1p2-11:35:012
3452235993523,9cyclictest210-21jbd2/mmcblk1p2-11:35:002
3452235993522,10cyclictest30-21ksoftirqd/209:00:022
3452235993520,12cyclictest3479903-21sshd11:45:542
345223399352,5cyclictest131rcu_preempt12:30:011
3452233993521,10cyclictest23-21ksoftirqd/107:40:001
3452227993527,5cyclictest16950irq/224-800000009:43:550
345223999343,3cyclictest131rcu_preempt09:25:013
3452239993425,5cyclictest3478111-21munin-plugin-st11:15:003
3452239993424,7cyclictest37-21ksoftirqd/307:15:013
3452239993424,5cyclictest309-21in:imuxsock12:25:003
3452239993422,10cyclictest37-21ksoftirqd/310:10:013
3452239993421,9cyclictest1-21systemd09:50:023
345223599345,5cyclictest3453459-21latency_hist07:35:032
345223599343,4cyclictest131rcu_preempt07:50:032
3452235993427,4cyclictest3482186-21latency_hist12:30:022
3452235993426,5cyclictest3456488-21latency_hist08:30:012
3452235993424,8cyclictest30-21ksoftirqd/209:25:012
3452235993423,7cyclictest3459981-21unin-run09:40:022
3452235993421,9cyclictest1-21systemd10:05:012
345223599342,18cyclictest0-21swapper/212:25:022
345223399342,3cyclictest131rcu_preempt11:30:011
345223399342,3cyclictest131rcu_preempt10:05:001
345223399342,3cyclictest131rcu_preempt08:40:021
345223399342,3cyclictest131rcu_preempt07:15:031
3452233993420,11cyclictest275-21dbus-daemon11:50:011
3452227993422,9cyclictest12-21ksoftirqd/008:25:010
3452239993324,6cyclictest3452959-21latency_hist07:25:023
3452239993320,9cyclictest3482461-21sed12:35:033
3452239993320,9cyclictest3482461-21sed12:35:033
3452235993325,5cyclictest3479421-21sendmail11:40:002
3452235993325,5cyclictest3455789-21sendmail08:20:002
3452235993322,8cyclictest309-21in:imuxsock09:04:462
3452235993322,8cyclictest309-21in:imuxsock09:04:452
3452235993321,8cyclictest30-21ksoftirqd/209:45:012
3452235993320,9cyclictest238-21systemd-journal08:00:012
3452235993320,10cyclictest3453567-21sort07:35:022
345223399334,3cyclictest131rcu_preempt08:55:021
345222799334,9cyclictest12-21ksoftirqd/007:30:000
3452239993222,6cyclictest3458822-21latency_hist09:15:023
3452239993220,8cyclictest275-21dbus-daemon08:35:013
3452239993219,10cyclictest3460202-21sshd09:44:553
345223999321,4cyclictest131rcu_preempt09:55:003
345223599322,4cyclictest131rcu_preempt09:50:012
3452235993221,8cyclictest30-21ksoftirqd/210:35:022
3452235993219,10cyclictest30-21ksoftirqd/211:40:022
3452235993215,14cyclictest275-21dbus-daemon08:25:012
345223999315,4cyclictest131rcu_preempt10:15:003
345223999312,3cyclictest131rcu_preempt11:30:023
3452239993122,6cyclictest3467205-21modprobe10:54:593
3452239993120,8cyclictest238-21systemd-journal08:40:543
3452239993120,8cyclictest238-21systemd-journal08:40:543
3452235993124,4cyclictest3452307-21latency_hist07:10:022
3452235993114,14cyclictest3452692-21systemctl07:19:592
345223599303,3cyclictest131rcu_preempt09:50:532
345223599301,9cyclictest3452567-21latency_hist07:15:032
3452235993015,12cyclictest1-21systemd09:35:012
3452235993011,5cyclictest131rcu_preempt09:10:012
3452239992919,7cyclictest3479140-21cron11:35:013
3452239992919,7cyclictest3479140-21cron11:35:003
345223399291,3cyclictest131rcu_preempt11:45:001
3452239992814,11cyclictest1-21systemd08:25:003
3452235992817,8cyclictest1-21systemd07:45:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional